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ABo TRACT 


A  detailed  analysis  was  made  of  the  tasks  performed  by  the  pilot 
of  a  modern  airborne  weapon  system.  The  results  of  this  analysis  were 
coded  and  presented  in  diagrammatic  form  to  show  alternative  courses  of 
action  that  may  result  in  successful  completion  of  a  mission  phase. 
Instances  of  pilot  decision  making  were  identified  and  a  way  of  concep¬ 
tualizing  decision  making  so  as  to  encompass  these  instances  was  pro¬ 
posed  . 


The  more  prominent  decision  theories  were  reviewed  briefly  and 
the  applicability  of  each  theory  to  the  problem  of  pilot  decision 
making  was  considered.  Pr-oblem  areas  requiring  experimental  study  were 
discussed  and  some  approaches  to  the  study  of  these  problems  were 
suggested . 
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PREFACE 


The  general  problem  area  within  which  this  study  program  origi¬ 
nated  is  presented  below  as  stated  in  the  contract  statement  of  work. 

"From  extrapolation  of  the  past  and  present  state-of-the-art 
developments  in  Air  Force  Weapons  Systems,  it  can  anticipated  that 
the  weapons  systems  of  the  near  future  (1970  era)  will  be  primarily 
controlled  by  automatic  equipment.  However,  it  is  also  anticipated 
that  many  of  these  weapon  systems  will  be  manned  in  order  to  provide 
flexibility  of  the  mission  profiles.  In  such  aircraft  the  role  of  the 
man  will  be  primarily  that  of  a  monitor  and  flight  program  selector. 
Thus,  the  requirements  for  cockpit  instrument  displays  will  be  differ¬ 
ent  from  those  found  in  present-day  aircraft.  In  future  aircraft  the 
informational  displays  should  be  designed  to  maximize  (1)  the  ability 
of  human  operators  to  monitor  the  aircraft  flight  parameters,  and  (2) 
the  ability  of  human  operators  to  make  appropriate  decisions  relating 
to  flight  programming." 

In  accordance  with  the  reference  date  (1970  era)  mentioned  in  the 
preceding  paragraph,  the  study  program  was  given  the  title  "Cockpit 
Instrument  Displays  for  the  1970  Era."  The  authors  believe  that  the  use 
of  this  title  for  the  present  report  would  be  misleading  for  two  reasons. 
One  of  these  is  concerned  with  the  nature  of  the  contents  of  the  re¬ 
port,  and  the  other  is  concerned  with  the  reference  date  of  1970. 

A  study  of  the  "decision-making"  role  of  the  human  operator  was 
considered  to  be  the  necessary  first  step  in  attacking  the  general 
problem  area.  The  study  was  begun  by  performing  a  detailed  analysis  of 
the  tasks  required  of  a  pilot  in  the  operation  of  a  complex  weapon  sys¬ 
tem.  The  results  of  this  analysis  were  organized  in  a  manner  that  per¬ 
mitted  identification  of  instances  of  pilot  decision  making.  These  ex- 
REples  of  decision  making  were  then  studied  in  detail  and  a  concept  of 
daclsion  making  believed  adequate  to  subsume  them  was  formulated. 
Following  this,  a  survey  of  decision  theories  was  made  to  determine 
which,  if  any,  might  be  profitably  applied  to  pilot  decision  making  as 
it  had  been  conceptualized.  The  present  report  contains  a  description 
of  the  procedure  and  the  results  of  this  work.  Consequently,  although 
the  study  program  was  generated  as  a  result  of  anticipated  display  de¬ 
sign  problesis,  the  report  is  concerned  with  the  study  of  the  human 
function  for  which  displays  are  required  rather  ^hau  with  the  displays 
ther  isel  ves . 

The  implications  of  the  results  of  the  study  of  pilot  decision 
making  are  considered  by  the  authors  not  to  be  specific  to  manned  weap¬ 
on  systems  of  the  1970  era,  but  rather  to  be  general izable  to  any  future 
Bsanned  weapon  systems.  In  this  connection,  it  may  be  pointed  out  that 
recent  developments  in  the  conquering  of  space  flight  problems  by  the 
United  States  and  Russia  have  had  the  effect  of  emphasizing  the  impor¬ 
tance  of  an  understanding  of  the  decision-making  role  of  the  human 
operator.  There  are  good  indications  that  one  of  the  goals  of  the  space 
flight  programs  of  our  own  country  and  other  countries  is  to  put  man 
into  space.  In  all  manned  systems,  with  the  possible  exception  of 
early  experimental  systems  whose  specific  purpose  may  be  merely  to 
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study  human  physiological  responses  to  space  flight  conditions,  the 
human  will  be  required  to  perform  those  functions  for  which  his  capa¬ 
bilities  surpass  those  of  automatic  devices.  Man>  of  these  functions 
are  included  under  the  general  heading  of  "decision-making."  There¬ 
fore,  although  the  title  of  the  study  program  refers  to  the  1970  era, 
the  contents  of  the  report  concerned  with  the  role  of  the  man  as  a 
decision  maker  are  not  limited  to  any  such  specific  time  period,  par¬ 
ticularly  any  such  period  in  the  relatively  near  future. 
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INTRODUCTION 


Throughout  the  foreseeable  future  there  will  be  manned  military 
aircraft  and  there  may  be  manned  space  vehicles.  The  tasks  to  be  per¬ 
formed  by  men  in  these  systems  will  be  deteimined  when  the  systems  are 
designed  and  will  depend  upon  the  nature  of  the  mission  to  be  accom¬ 
plished  and  upon  the  state  of  the  designer’s  art.  Judging  from  the 
history  of  the  development  of  manned  airborne  weapon  systems  it  may  be 
anticipated  that  the  tasks  performed  by  men  in  future  weapon  systems 
will  be  somehow  different  from  the  tasks  curren  ly  being  performed  by 
system  operators. 

There  is  no  doubt  that  up  to  the  present,  at  least,  the  task  of 
the  aircrew  has  changed  with  time.  For  one  thing,  it  has  grown  in  com¬ 
plexity,  If  complexity  can  be  measured  by  the  number  of  information 
channels  into  and  out  fror^  the  cockpit,  then  to  establish  such  a  growth 
one  need  only  compare  the  cockpit  of  a  present  day  fighter-interceptor, 
for  example,  with  those  of  its  counterparts  of  ten,  twenty,  thirty  and 
forty  years  ago,  A  second  measure  of  complexity  might  be  the  number  of 
formal  training  hours  deemed  necessary  for  a  crew  member  to  reach  pro¬ 
ficiency,  These  hours  have  also  increased  over  the  years. 

The  aircrewman's  task  has  changed  qualitatively  over  the  years. 

In  the  early  days  of  military  flying  the  pilot,  for  example,  was  mainly 
engrossed  in  the  manual  flight  control  of  his  aircraft.  It  was  not 
necessary  to  spend  much  time  with  the  "housekeeping"  controls  of  the 
aircraft.  The  fire  control  system  demanded  little  attention  and  com¬ 
munications  were  rudimentary.  As  these  and  other  system  functions  be¬ 
came  increasingly  complex  and  sophisticated  the  pilot  found  that  it  was 
necessary  to  spend  more  and  more  time  with  them.  Flying  the  airplane 
became  just  one  among  many  tasks  to  be  performed.  At  the  present  time 
the  latest  interceptor  design  requires  almost  no  manual  flying  by  the 
pilot.  An  automatic  flight  control  system  flies  the  aircraft  from  short¬ 
ly  after  takeoff  to  just  before  touchdown.  The  pilot  Is  not  idle  however. 
He  is  busily  ‘’■nga '•ed  in  the  operation  of  the  various  aix'Craft  subsystems, 
choosing  modes  of  operation,  communicating,  monitoring  the  actual  flight 
of  the  aircraft  and  making  tactical  "decisions"  where  necessary.  This 
pilot’s  task  is  qualitatively  very  different  from  those  of  his  pi'ede- 
cessors. 


A  number  of  points  can  bo  made  from  this  very  brief  resv.une  of 
history.  The  first  is  that  the  aircrewman ' s  task  has  changed  because 
equipment  has  been  addejl  to  the  system.  Operator  equipment  lias  been 
added  sometimes  in  a  direct  effort  to  increase  system  capability,  some¬ 
times  as  a  consequence  of  an  increase  in  capability  achieved  another 
way  and,  occasionally  in  an  effort  to  simplify  the  operator's  task. 

Adding  a  radar  set  to  an  interceptor  is  an  example  of  an  attempt 
to  achieve  a  direct  increase  in  capability  for  the  system.  Adding 
special  engine  controls  such  as  cowl  flaps  or  afterburner  is  a  consequei-ce 
of  building  engines  yielding  higher  system  performance.  Providing  an 
autopilot  is  an  example  of  adding  equipment  to  simplify  the  pilot's  task, 


MaauBcrlpt  i'^leassd  by  tlie  authors  for  publication  as  a  WADC  Techni¬ 
cal  Keport  in  October  1958. 

WADC  TR  58-522 


1 


and  thereby  changing  it. 


Turning  now  to  the  future  there  is  every  reason  to  suppose  that 
equipment  will  continue  to  change.  It  can  be  expected  to  increase  in 
quantity  as  new  functions  and  capabilities  are  added  to  systems.  To 
offset  this  trend  there  is  the  considerable  recent  interest  and  activity 
in  devising  means  to  simplify  the  operator's  task.  This  is  becoming 
manifest,  for  example,  in  the  many  computational  aids  being  provided 
operators  in  new  systems,  in  automatic  control  devices,  and  in  the  de¬ 
sign  of  integrated  displays  and  controls  for  operator  use.  All  these 
trends  taken  together  simply  indicate  that  the  operator's  task  in  the 
future  will  almost  surely  be  different  than  it  is  today  if  manned  sys¬ 
tems  continue  to  be  used  as  first  line  tactical  vehicles.  The  question 
of  manned  versus  unmanned  vehicles  is  not  the  subject  of  this  report 
although  the  question  will  be  raised  again  in  a  different  context.  At 
present,  there  appear  to  be  definite  plans  for  manned  systems,  both 
aircraft  and  satellite-type  vehicles,  that  will  extend  them  well  into 
the  future. 


It  is  assumed,  therefore,  that  (1)  there  will  bo  manned  systems 
of  interest  in  the  future,  and  (2)  the  tasks  perf ora.u]  by  the  men  will 
be  in  some  way  different  from  those  now  performed  in  manned  systems. 


What  will  this  difference  be?  Again  turning  to  an  extrapolation 
of  past  trends  it  will  be  recalled  that  in  the  earlier  days  of  flying 
the  pilot,  for  example,  appeared  to  be  almost  completely  occupied  with 
a  continuous  manual  control  task,  i.e,,  manipulating  the  aircraft  flight 
controls  in  such  a  way  as  to  cause  the  aircraft  to  make  good  some  flight 
path  he  had  selected,  usually  by  visual  reference  to  the  outside  world. 
In  the  latest  interceptor  on  the  other  hand  the  pilot  is  relieved  al¬ 
most  entirely  of  the  continuous  manual  control  task  and  instead  spends 
his  time  looking  at  instrument  displays  and  turning  switches,  knobs, 
and  other  controls  that  were  not  even  to  be  found  in  the  cockpits  of 
former  aircraft. 


A  second  difference  is  that  the  earlier  pilot  chose  his  flight 
path  largely  by  direct  view  of  the  outside  world.  He  could  see  immedi¬ 
ately  where  he  wanted  to  go  with  respect  to  the  ground  or  with  respect 
to  the  target.  The  modern  pilot  obtains  only  a  small  amount  of  infor¬ 
mation  from  this  source;  most  of  his  information  comes  from  artificial 

one  sort  or  anothei'  within  th  cockpit.  The  I'uason  for  this 
information  requii'cd  to  fly  a  modern  mission  simply  cannot 
by  direct  view  of  the  outside  world.  Targets  must  be  dc- 
they  are  too  fai*  away  to  be  seen  ov  ■when  weather  prevents 
seen.  Flight  paths  must  be  clu'sen  with  more  accuracy  than 
can  be  ■judged  and  instead  must  1.  precisely  computed.  An  organi^.od 
defense' or  attack,  or  simply  adequate  traffic  control,  may  require  that 
information  be  obtained  from  widely  scattered  sources,  assembled,  organ¬ 
ized,  and  sent  in  the  form  of  commands  to  the  various  par ticipatitig 
aircraft  —  a  feat  that  could  not  bo  accomplished  through  the  medium  of 
the  individual  pilot's  own  sense  organs  acting  independently.  Thus, 
the  modern  pilot  receives  information  originating  beyond  the  range  or 
outside  the  bandwidth  of  his  own  perceptual  mechanism.  He  depends 
largely  upon  information  of  this  typo  as  presented  by  numerous  instrument 
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displays  and  acts  ia  accordan^a  with  his  interpretations  of  what  is 
presented.  The  earlier  pilot  did  not  do  this. 

A  third  difference  between  the  two  tasks  is  found  in  what  might 
be  described  as  the  sophistication  of  modern  systems  compared  with  older 
systems ,  The  modern  complex  weapon  system  normally  offers  the  aircrew 
alternative  modes  of  operating  whereas  the  older  systems  did  not,  or  at 
least  only  to  a  very  limited  extent.  Alternative  modes  are  provided  for 
two  reasons.  They  are  provided  as  back-up  modes  in  case  of  equipment 
failure  in  the  primary  mode.  They  are  provided  in  order  to  counteract 
events  that  might  happen  during  a  mission,  perhaps  by  enemy  action,  and 
that  would  seriously  degrade  the  probability  of  mission  success.  An 
example  of  the  former  is  the  provision  for  direct  manual  control  of  the 
aircraft  in  case  of  failure  in  the  automatic  pilot  where  the  autopilot 
is  the  pr-'mary  mode  of  flight.  An  example  of  the  latter  is  the  provision 
for  alternative  armament  loadings  as  a  hedge  against  various  enemy  counter- 
measures.  A  modern  weapon  system  may  offer  up  to  thirty  or  thirty-five 
modes  o?  operation  of  the  second  kind  — not  all  of  equal  significance. 

Older  weapon  systems  seldom  offered  more  than  three  or  four.  The  choice 
among  modes  is  normally  made  an  operator  function.  Hence  today's  oper¬ 
ator  has  many  more  choices  to  make  than  yesterday’s.  This  is  reflected 
in  the  cockpits  of  each  system  — today's  with  many  switches  and  controls, 
yesterday's  with  relatively  few. 

A  reasonable  extrapolation  of  the  past  trend  in  operator  tasks  to 
the  future  suggests  that  operators  will  become  less  and  less  concerned 
with  continuous  manual  control  tasks  and  more  and  more  concerned  with 
the  interpretation  of  information  assembled  from  a  variety  of  sources 
and  displayed  "artificially"  within  the  cockpit  and  with  the  choice  of 
operating  mode  based  upon  the  information  received.  This  kind  of  activ¬ 
ity  corresponds  closely  with  what  is  commonly  known  as  the  exercise  of 
Judgement  or  the  making  of  decisions. 

With  this  back-ground  the  present  task  was  undertaken.  It  was  felt 
thiit  while  the  words  "decision  making"  might  generally  describe  what 
operators  of  weapon  systems  of  the  future  could  be  expected  to  spend 
much  of  their  time  doing,  there  e.xisted  no  firm  understanding  of  what 
the  words  really  mean.  When  it  is  said  that  an  ope'rator  makes  a  decision, 
what  does  he  really  do?  What  implications  do  his  actions  have  for  the 
design  of  the  system,  for  defining  his  information  requirements,  for 
Specifying  the  necessary  displays  and  controls?  ffhat  human  engineering 
research  t-hould  be  undertaken  now  in  anticipation  of  the  new  tasks? 

While  il  was  not  expected  that  definitive  answers  could  be  found 
for  all  these  questions,  it  was  agreed  that  it  would  be  worthwhile  at 
least  to  attack  the  problem,  and  this  is  what  has  been  done. 

The  plan  of  attack  was  to  proceed  from  the  known  to  the  unknown 
via  whatever  help  existing  theory  could  provide.  The  known  in  this  case 
was  chosen  to  be  a  new  electronic  and  control  system  ui^  Tn  interceptor 
aircraft.  It  is  Just  at  the  time  of  writing  going  into  prodvictioa.  It 
is  a  sophisticated  system  and  one  in  which  tno  operator  plays  a  major 
role.  It  was  felt  that  if  the  operator's  task  in  this  system  could  be 
analysed  in  detail,  considerable  information  could  be  gained  about 


Wjilfx;  TR  f>8-522 


3 


operator  decision  making, at  least  in  currently  advanced  systems. 

Next  it  was  hoped  to  organize  this  material  in  such  a  way  that 
extrapolation  to  unknown  future  systems  could  be  made.  In  this  connec¬ 
tion  the  plan  was~lo  review  current  human  decision  theory,  on  whicn  there 
is  an  extensive  literature,  to  see  if  it  could  be  applied  to  the  opera¬ 
tor's  problem,  thus  aiding  in  the  extrapolation. 

Finally  it  was  the  plan  to  speculate  to  some  extent  on  answers  to 
some  of  the  questions  asked  above. 
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OPERATOR  DECISION  ANALYSIS  FOR  A  KNOWN  SYSTEM 


In  the  past  the  words  "decision  making"  have  been  used  to  refer 
to  what  the  man  does  in  situations  that  appear  to  be  quite  diverse.  As 
examples,  in  the  context  of  an  airborne  weapon  system  mission,  it  has 
been  assumed  that  the  man  makes  a  decision  both  when  he  discriminates  a 
target  echo  on  a  radar  scope  and  when  he  selects,  prior  to  takeoff,  a 
flight  program  from  among  many  possible  alternative  flight  programs. 

The  decision  process  has  been  conceptualized  somewhat  differently  in 
the  development  of  theoretical  models  for  situations  that  are  analogous 
to  the  two  airborne  situations  given  here  as  examples. 

One  way  of  attacking  the  problem  of  pilot  decision  making  would  be 
to  study  the  applicability  of  various  existing  models  to  specific  instan¬ 
ces  of  what  has  been  termed  decision  making  in  the  airborne  situation. 
Rather  than  do  this,  however,  it  was  considered  desirable  first  to  search 
for  a  way  of  conceptualizing  decision  making  that  would  encompass  the 
man’s  behavior  in  all  situations  in  which  it  is  ordinarily  assumed  that 
he  makes  decisions. 

This  kind  of  approach  to  the  problem  required  a  detailed  study  of 
a  manned  airborne  weapon  system  mission  in  terms  of  the  tasks  performeo 
by  the  man.  The  weapon  system  chosen  foi“  this  mission  analysis  was  the 
F-106/MA-1.  The  first  objective  was  simply  to  study  the  weapon  system 
mission  and  collect  as  much  information  as  possible  about  the  tasks  the 
m'in  performs.  An  attempt  v/as  made  to  avoid,  insofar  as  possible,  the 
influence  of  any  preconceived  notions  about  the  nature  of  the  decision 
process.  Also,  during  the  early  stages  of  the  analysis  there  was  no 
concern  about  a  criterion  or  criteria  for  identifying  instances  of  de¬ 
cision  making. 

In  order  to  simplify  the  task  of  analysis,  the  first  effort  was 
concerned  with  an  idealized  noi’mal  mission,  A  list  was  made  of  the  tasks 
the  pilot  has  to  perform  assuming  that  the  system  functions  properly, 
the  enemy  obligingly  appears  at  the  proper  time  and  utilizes  no  countei- 
measures.  etc.  This  comprehensive  list  covered  the  pilot  s  activities 
from  the' time  he  enters  the  cockpit  for  ground  checks  prior  to  takeoff 
until  he  returns  to  the  I'ainp  after  the  auccessfal  completion  of  the  mis¬ 
sion. 

The  list  was  then  expanded  by  adding  to  it  the  tasks  that  would  be 
required  if  certain  events  such  as  system  malfunctions  and  enemy  countei- 
measures  were  to  occur  during  the  iBission. 

The  next  stage  of  the  program  consisted  of  attempts  to  incorpo¬ 
rate  this  information  in  flow  diagrams  representing  the  alternative 
sequences  of  events  that  might  occur  during  a  mission.  A  major  prob¬ 
lem  encountered  at  this  point  was  that  of  developing  a  system  ol  coding 
the  various  kinds  of  events  for  diagrammatic  repr  -sentation .  Also,  it 
was  immediately  apparent  that  it  would  be  practically  impossible  to  show 
all  of  the  possible  alternative  sequences  of  events  that  might  occur. 

Even  when  only  a  relatively  small  number  of  the  possible  adverse  events 
that  could  be  antic ip..tod  were  considered,  the  diagrammatic  representation 
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became  extremely  intricate. 

A  number  of  schemes  for  coding  the  information  and  simplifying 
the  diagram  were  investigated.  Although  the  different  schemes  varied 
in  certain  respects  they  all  involved  essentially  the  same  procedural 
approach.  In  each  case  the  diagrammatic  representation  was  constructed 
by  starting  with  a  set  of  conditions  at  an  arbitrary  point  (either  at 
takeoff  or  at  the  beginning  of  some  particular  phase  of  the  mission). 

Then  an  attempt  was  made  to  show  the  various  possible  alternative  se¬ 
quences  of  events  by  successive  branchings  of  the  lines  representing 
ongoing  courses  of  action.  The  results  were  largely  unsatisfactory  so 
this  procedure  was  abandoned. 

The  next  approach  that  was  tried  involved  a  procedure  that  was 
somewhat  the  reverse  of  the  one  described  above.  An  attempt  was  made 
to  construct  a  diagram  by  starting  with  a  relatively  small  number  of 
different  states  of  the  system  and  then  working  backward,  so  to  speak, 
specifying  the  events  antecedent  to  these  states. 

The  states  of  the  system  that  were  chosen  for  the  starting  points 
in  this  procedure  were  those  that  characterize  a  successful  attack  phase 
of  the  mission.  More  specif ijally,  they  were  those  states  from  which 
armament  may  be  fired  successfully.  The  states  are  defined  by  the 
various  permissible  combinations  of  such  factors  as  attack  geometries, 
radar  modes,  and  types  of  armament  selected. 

Although  the  manner  of  devising  this  diagram  involved  working 
backward,  the  resulting  symbolic  structure  was  one  that  represented 
the  successive  occurrence  of  events  in  a  conventional  fashion.  The 
diagram  presented  a  clear  indication  of  how  the  system,  starting  in  a 
specified  state  at  some  arbitrary  point,  can  eventually  get  into  a  mode 
of  opei’ation  resulting  in  one  of  the  system  states  that  define  a  success¬ 
ful  attack. 

In  order  to  keep,  the  diagram  as  simple  as  possible,  the  sequences 
of  events  were  traced  back  through  their  respective  points  of  origin  to 
a  single  common  origin  arbitrarily  located  jv-st  prior  to  the  time  of 
reaching  the  offset  point*.  Thus,  the  diagram  in  general  represented 
the  alternate  ways  in  which  the  attack  phase  of  a  mission  may  be 
accompli  shed. 

Three  categories  of  events  were  included  in  the  diagram.  These 
were  (1)  infoi’mation  received  by  the  pilot  (usually  visually),  (2) 
actions  taken  by  the  pilot,  and  (3)  actions  carried  out  automatically 
by  the  system.  For  the  sake  of  convenience  these  events  may  be  re¬ 
ferred  to  as  (1)  pilot  sees,  (2)  pilot  does,  and  (3)  system  does, 
respectively. 

With  this  kind  of  representation  of  the  sequences  of  events  that 


♦The  offset  point  is  a  navigation  point  from  which  f.he  target  may 
be  expected  to  be  seen  by  the  search  radar  and  signifies  the  start  of 
the  attack  phase  of  the  mission. 
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may  occur  during  the  attack  phase  of  the  mission  available,  the  next 
step  was  one  of  identifying  the  points  where  it  can  be  said  that  the 
pilot  makes  decisions.  Without  making  any  committment  as  to  tie  nature 
of  the  decision  process  itself,  it  was  agreed  that  the  pilot  makes  a 
decision  each  time  an  ongoing  course  of  action  branches  into  alternate 
courses  of  action  as  a  result  of  the  pilot's  behavior.  At  eacn  of  these 
points  on  the  diagram  the  following  information  was  available:  (1)  the 
nature  of  the  ongoing  course  of  action,  (2)  the  information  that  is  pre¬ 
sented  to  the  pilot  when  an  event  occurs  that  renders  this  course  of 
action  no  longer  appropriate  for  successful  completion  of  the  mission, 

(3)  the  alternative  course (s)  of  action  that  are  appropriate,  (4)  the 
things  the  pilot  does  to  change  the  mode  of  system  operation  to  carry 
out  an  alternative  course  of  action,  and  (5)  the  expected  results  of 
each  alternative  course  of  action. 

Study  of  the  points  where  an  ongoing  course  of  action  branches 
into  alternate  courses  of  action  led  to  a  concept  of  decision  making 
somewhat  different  from  the  one  ordinarily  advanced.  It  was  noted  that 
whenever  an  event  occurs  that  renders  an  ongoing  course  of  action  in¬ 
appropriate,  the  pilot  is  presented  with  an  indication  of  this.  If  one 
kind  of  indication  is  presented,  the  pilot  does  something  specific  in 
response  to  that  indication.  If  some  other  indication  is  presented,  he 
does  something  else.  In  other  words,  for  each  different  indication  there 
is  a  prescribed  appropriate  course  of  action.  The  conclusion  was  reached 
that  it  would  be  profitable  to  conceive  of  decision  making  in  terms  of 
discriminating  stimuli  rather  than  in  terms  of  selecting  responses.  For 
the  type  of  semi-automatic  system  under  consideratic.  ^  it  seemed  that 
when  the  pilot  makes  a  decision  he  is  not  deciding  "What  response  should 
I  make?",  but  rather  he  is  deciding  "What  is  the  state  oi  the  system?" 
Once  he  has  diagnosed  the  state  of  the  system  the  appropriate  response 
is  known.  The  state  of  the  system  specifies  the  response  to  be  made. 

How  this  way  of  conceptualizing  the  decision  process  was  developed 
can  be  further  clarified  by  describing  in  some  detail  actual  sequences 
of  events  in  a  diagram  representing  the  attack  phase  of  the  mission.  In 
Figure  1  the  various  events  are  coded  in  terms  of  (1)  what  the  pilot  sees 
(circles),  (2)  what  the  pilot  does  (rectangles  with  rounded  corners),  and 
(3)  what  the  system  does  automatically  (rectangles).  A  shaded  rectangle 
indicates  a  course  of  action  carried  out  automatically  resulting  in  suc¬ 
cessful  armament  firing,  A  cross-hatched  rectangle  indicates  that  an 
attack  is  broken  off. 

Since  some  of  the  events  represented  by  the  circles  and  rectangles 
are  concerned  with  modes  of  system  operation  that  are  classified,  the 
events  represented  by  the  symbols  are  identified  only  in  general  terms. 
For  purposes  of  illustrating  the  nature  of  pilot  decision  making  in  this 
kind  of  system,  however,  detailed  knowledge  of  the  nature  of  specific 
events  is  unnecessai’y .  The  interest  is  not  in  the  details  of  the  events 
themselves,  but  rather  in  the  relationships  among  the  three  general 
classes  of  events.  The  diagram  is  to  be  interpreted  as  follows: 

The  attack  is  considered  to  have  its  starting  point  at  an  arbi¬ 
trary  moment  in  time  just  after  turning  through  the  offset  point.  At 
this  time  the  pilot  sees  (No.  1  on  the  diagram)  on  the  radar  scope  the 
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range  sweep,  antenna  elevation  marker,  target  elevation  marker,  target 
marker  circle,  steering  dot  and  reference  circle,  Assuming  that  the 
pilot  has  not  previously  received  information  leading  him  to  expect 
other  than  normal  attack  conditions  he  adjusts  the  azimuth  scan,  arti¬ 
ficial  horizon,  and  antenna  elevation  on  the  radar  scope  (2).  He  may 
then  see  a  target  echo  o.:  the  radar  scope  (3).  Ee  depresses  the  action 
trigger  (4) ,  and  the  range  gate  marker  appears  on  the  scope  (5) .  He  ad¬ 
justs  the  radar  antenna  hand  control  (6).  The  target  echo  and  range 
gate  marker  merge,  followed  by  the  appearance  of  the  attack  display  on 
the  radar  scope  (7).  The  pilot  releases  the  action  trigger  (8)  and  the 
system  automatically  executes  the  attack  (9).  Although  the  attack  is 
executed  automatically  by  the  system,  information  about  the  progress  of 
the  attack  is  continuously  displayed  to  the  pilot  (10),  At  the  proper 
time  the  system  fires  the  armament  automatically  (11)  and  a  signal  indi¬ 
cating  the  occurrence  of  this  event  appears  on  the  radar  scope  (12), 

It  should  be  noted  that  under  the  conditions  of  a  normal,  ideal 
attack  assumed  above,  the  sequence  of  events  occurring  prior  to  initia¬ 
tion  of  the  automatic  flight  program  consists  of  an  alternation  of  "pilot 
sees"  and  "pilot  does"  types  of  events.  In  other  words,  a  noi’mal  se¬ 
quence  requires  no  "decisions"  in  the  sense  of  selecting  a  course  of 
action  from  among  several  alternatives.  As  a  result  of  his  training  the 
pilot  makes  responses  appropriate  to  events  as  he  is  informed  of  the 
occurrence  of  these  events  by  his  displays.  His  "decisions"  are  in  the 
nature  of  diagnoses  of  the  state  of  the  system,  rather  than  selections 
of  courses  of  action. 

Let  us  now  consider  the  pilot's  performance  during  a  mission  that 

departs  from  the  "ideal"  normal  mission  as  a  result  of  the  occurrence  of 

some  unpropitious  event.  Consider  that  this  event  is  the  use  of  counter¬ 
measure  A  by  the  enemy  sometime  following  the  initiation  of  the  automatic 
flight  program.  The  pilot  sees  evidence  of  this  countermeasure  on  his 
radar  scope  (13),  He  performs  counter-countermeasure  a  (14)  and  the  sys¬ 
tem  automatically  executes  the  attack  (15).  If  no  addrtional  adverse 

events  occur,  the  sequence  is  continued  through  automatic  firing  (11) 

and  the  displayed  indication  of  firing  (12), 

As  was  the  case  in  the  normal  attack  sequence,  the  pilot's  actions 

are  specified  by  the  information  displayed  to  him.  Once  he  has  deter¬ 

mined  the  state  of  the  system  (in  this  case,  the  presence  of  enemy 
countermeasure  A)  the  response  he  should  make  is  also  determined.  The 
latter  is  flxed”by  the  design  f  the  system  and  by  the  training  in  the 

use  of  the  system  that  the  pilot  has  received. 

We  may  also  consider  the  possibility  that  there  is  a  cessation  of 
enemy  countermeasure  A  after  counter-countermeasure  a  has  been  acconi- 
plished  and  the  automatic  flight  program  initiated.  The  pilot  sees  evi¬ 
dence  of  the  cessation  of  countermeasure  A  on  the  radar  scope  (17) , 

This  may  be  followed  either  by  the  appearance  of  a  target  echo  (18)  or 
no  target  echo  may  appear  (24).  If  the  pilot  correctly  decides  that 
there  is  a  target  echo  (18)  he  adjusts  the  radar  antenna  hand  control 
(19)  and  the  attack  display  appears  on  the  radar  scope  (20),  He  releases 
the  action  trigger  (21)  and  the  system  automatically  executes  the  attack 
(22)  which  again  terminates  in  automatic  firing  (11),  If  the  pilot 
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correctly  decides  that  no  target  echo  is  present  on  the  radar  scope  (24) 
he  presses  the  return-to-search  switch  (25)  and  the  attack  is  broken  off 
(26) . 

The  various  alternative  sequences  outlined  above  are  sufficient  to 
illustrate  the  concept  of  decision  making  arrived  at  through  study  of 
the  F-106/MA-I  weapon  system  mission.  Almost  invariably,  at  the  points 
in  the  mission  where  an  ongoing  course  of  action  branches  into  alterna¬ 
tive  coui'ses  of  action  as  the  result  of  the  occurrence  of  some  unfavor¬ 
able  event,  the  alternative  courses  are  differentiated  initially  by  in¬ 
formational  displays  that  are  different,  one  from  the  other.  This  leads 
to  the  Conclusion  that  it  should  be  profitable  to  conceive  of  the  de¬ 
cisions  made  by  the  pilot  not  in  terms  of  the  pilot's  selection  of  a 
course  of  action  from  among  the  alternatives  available,  but  rather  in 
terms  of  his  diagnosis  of  the  state  of  the  system.  In  other  words,  the 
pilot  does  not  choose  a  course  of  action  but  he  does  decide  what  is  the 
nature  of  the  situation.  Once  he  has  decided  what  the  state  of  the  sys¬ 
tem  is,  the  course  of  action  is  specified. 

In  the  F-106/MA-1  system  mission  an  exception  to  the  rule  that 
alternative  courses  of  action  are  differentiated  initially  by  different 
informational  displays  representing  different  states  of  the  system  is 
indicated  in  the  diagram  following  the  performance  of  event  (2)  by  the 
pilot.  If  the  enemy  uses  countermeasure  A  at  this  point,  evidence  of 
this  event  appears  on  the  pilot's  radar  scope  (27).  The  pilot  is  then 
confronted  with  the  selection  of  either  counter-countermeasure  ^  (31) 
or  counter-countermeasure  b  (28).  The  information  presented  on  the 
r,adar  scope  (27)  does  not  specify  which  of  the  two  responses  should  be 
made.  If  the  pilot  performs  counter-countermr'’-sure  b  (28)  the  target 
echo  may  appear  on  the  scope  (29)  or  it  may  not  appear  (30),  If  the^ 
pilot  correctly  decides  that  there  is  a  target  echo  on  the  scope  (29) 
he  then  presses  the  action  trigger  (4)  and  is  back  on  the  normal  se¬ 
quence  of  events.  On  the  other  hand,  if  he  correctly  decides  that  there 
if;  no  target  echo  on  scope  (30)  he  then  performs  counter-counter¬ 
measure  a  (31),  which  is  a  response  he  could  have  made  initially  when 
he  detected  the  occuvr^rcvg  of  countermeasure  A  (27)  . 

Thus,  the  alternatives  (28)  aud  (31'’  seem  to  characterize  a  situ¬ 
ation  of  the  type  ususll,''/  describee,  when  decision  making  is  conceived 
of  as  the  selection  of  a  coi-r-se  of  action  from  among  two  or  more  alter¬ 
natives  „ 

Actually,  countermeasure  A,  whose  occurrence  is  displayed  (27) 
may  be  either  one  of  two  couuterineasures ,  which  while  similar  in  nature, 
are  most  effectively  coped  with  by  means  of  two  different  counter-counter¬ 
measures.  The  display  may  not  in  all  cases  immediately  indicate  which 
of  the  two  related  counttfrmeasures  is  being  used.  The  pilot  has  the 
option  of  performing  counter-countermeasure  a  (31)  which  will  initiate 
a  course  of  action  that  will  counteract  either  of  the  two  countermeasures, 
or  of  performing  counter-countermeasure  b  (28)  which  will  effectively 
inform  him  which  of  the  two  countermeasures  is  being  used.  Also,  if  he 
performs  counter-counterEieasure  b  and  subsequently  sees  a  target  echo  (29) 
he  knows  that  he  made  the  appropriate  response  and  may  continue  with  the 
normal  sequence  (4,  5,  6,  etc.).  If,  howevei',  after  performing  counter- 
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countermeasure  b  (28)  he  cannot  see  a  target  echo  (30)  he  knows  that 
the  appropriate  response  is  to  perform  counter-countermeasure  a  (31), 
which  could  have  been  made  initially.  The  advantage  of  performing 
counter-countermeasure  b  initially  is  that,  if  it  is  appropriate,  the 
system  is  immediately  back  on  the  normal  sequence  of  events.  The  dis¬ 
advantage  of  performing  counter-countermeasure  b  initially  is  that  if 
it  is  inappropriate,  there  is  a  penalty  in  the  Torm  of  loss  of  time 
during  the  performance  of  counter-countermeasure  b  before  counter¬ 
countermeasure  a  is  eventually  made.  The  advantage  of  performing 
counter-countermeasure  a  initially  is  that  if  a  is  appropriate  the 
system  is  immediately  on  a  sequence'  of  events  T31>  32,  33,  et-’.)  which 
leads  to  the  initiation  of  an  automatic  attack  program.  The  disadvan¬ 
tage  of  performing  a  initially  is  that  while  a  is  effective  against 
both  countermeasures,  b  is  more  effective  witF  one  of  them,  and  if 
that  countermeasure  against  which  b  is  more  effective  actually  occurred, 
the  penalty  for  making  a  is  in  terms  of  a  relative  decrease  in  system 
performance. 

Fortunately  for  the  pilot,  the  situation  described  in  some  detail 
above  does  not  really  constitute  a  decision  problem.  The  probabilities 
of  occurrence  of  the  possible  outcomes  and  the  values  of  the  possible 
outcomes  for  each  of  the  alternatives  have  been  considered  by  those 
concerned  with  the  design  of  the  system  and  by  those  concerned  with 
planning  its  tactical  utilization.  The  decision  that  counter-counter¬ 
measure  b  (28)  should  always  be  made  first  when  there  is  evidence  of 
A-type  countermeasure  has  been  made  by  the  designers  and  tacticians. 
Thus,  even  though  the  pilot  knows  that  alternative  (31)  is  open  to  him 
when  (27)  is  displayed,  it  is  standard  operating  procedure  for  him  to 
perform  (28)  first. 

The  same  procedure  used  in  the  construction  of  the  diagram  rep¬ 
resenting  the  attack  phase  was  used  to  construct  a  diagram  of  the  alter¬ 
native  courses  of  action  that  might  be  carried  out  during  the  return- 
to-base  and  landing  phases  of  the  mission.  This  served  two  purposes. 
First,  it  provided  a  check  on  the  applicability  of  the  procedure  and 
coding  scheme  to  other  kinds  of  situations.  Second,  the  resulting 
diagram  permitted  a  check  on  the  generality  of  the  concept  of  decision 
making  arrived  at  through  study  of  the  attack  phase  diagram. 

The  diagram  in  Figure  2  presents  the  alternate  event  sequences 
for  the  return-to-base  and  landing  phases.  The  starting  points  for 
the  construction  of  this  diagram  were  those  states  of  the  system  from 
which  the  TRACAL  approach  control  area  may  be  entered.  For  the  purpose 
of  constructing  this  diagram,  - of  these  states  were  defined  by  the 
various  permissible  combinations  of  navigational  modes  and  flight  con¬ 
trol  modes. 

The  four  major  navigational  modes  considered  were  DL  MAX-RNG 
(data  link  maximum  range),  AUTO-N.AV  (automatic  navigation),  Map  Dis¬ 
play,  and  Visual  Contact.  Within  the  Map  Display  major  mode  there  are 
TACAN  and  ADF  sub-modes.  Within  the  ADF  sub-mode  there  are  three  sub- 
sub-modes  designated  ADF-DL,  ADF-CMD,  and  ADF-DL  and  CMD. 

The  three  flight  control  modes  considered  were  AUTO  (full  auto- 
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matic),  PA  (pilot  assist),  and  DMIPER  or  DIR  MAN  (direct  manual).  It 
should  be  noted  that  the  two  DAMPER  modes  (PITCH  AND  YAW)  '^nd  the  DIR 
MAN  mode  are  not  treated  separately  in  the  diagram.  The  a  itrary 
combination  of  these  three  flight  control  modes  permits  considerable 
simplification  of  the  diagram  and  does  not  appreciably  affect  the 
structure  of  the  diagram  insofar  as  pilot  decisions  are  concerned. 

The  combinations  of  navigational  modes  and  flight  control  modes 
defining  the  13  states  of  the  system  are  shown  in  Table  I. 
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LOSE  DATA  link 
LOSE  TACAN 

AUTO  NAV  unavailable 


LOSE  OAFA  link.  LOSE  TACAN 
AUTO  NAV  unavailable 
AOF  MALFUNCTION 


F  ioTOiMOC  ' 
AUTO  NAV 
kSticcT  Ho*iii*i  py 


The  antecedents  to  each  of  the  states  were  traced  back  to  a 
coimnon  origin  just  after  the  recovery  from  armament  launch  terminating 
the  attack  phase.  Following  the  recovery  from  the  attack  the  pilot 
contacts  GCI  for  instructions  concerning  either  reattack  or  return  to 
base. 


Although  it  is  possible  under  certain  circumstances  to  change 
directly  from  any  one  of  the  return- to-base  courses  of  action  to  any 
other  one  of  the  12  courses  shown  on  the  diagram,  nc i  all  of  the  pos¬ 
sible  pathways  for  doing  so  are  indicated.  The  rules  used  for  deter¬ 
mining  which  pathways  were  shown  on  the  diagram  are  as  follows: 

1.  Within  each  of  the  navigational  modes,  pathways  are  shown 
which  indicate  how  to  change  from  any  flight  control  mode  to  any  other 
flight  control  mode. 

2.  Among  the  different  navigational  modes,  the  pathways  shown 
are  those  which  indicate  how  to  change  from  any  navigational  mode  to 
an  equally  sophisticated  mode  or  to  a  less  sophisticated  mode.  For 
the  purpose  of  constructing  this  diagram  the  DL  and  AUTO  NAV  modes  were 
considered  to  be  equally  sophisticated.  Both  of  these  modes  were  con¬ 
sidered  to  be  more  sophisticated  than  the  Map  Display  mode,  which  is, 
in  turn,  more  sophisticated  than  the  Visual  Contact  mode.  Within  the 
Map  Display  mode  the  TACAN  sub-mode  was  considered  more  sophisticated 
than  the  ADF  sub-mode.  The  three  ADF  sub-sub-modes  were  considered 

to  be  equally  sophisticated. 

3.  For  changes  from  one  navigational  mode  to  another,  the  only 
pathways  shown  are  those  which  indicate  a  change  from  the  most  sophis¬ 
ticated  flight  control  mode  within  a  navigational  mode  to  the  various 
flight  control  modes  within  another  navigational  mode.  For  the  purpose 
of  constructing  this  diagram  the  AUTO  mode  was  considered  to  be  the 
most  sophisticated.  The  next  most  sophisticated  was  PA  (pilot  assist) 
which  was  followed  by  the  DAMPKR-DIR  MAN  combination. 

Table  II  summarizes  the  pathways  among  the  various  courses  of 
action  that  are  shown  in  the  return-to-basc  phase  diagram.  An  .K  in  a 
cell  indicates  reciprocal  pathways  between  the  courses  of  action  indi¬ 
cated  in  the  column  and  row  headings.  An  arrow  in  a  cell  indicates 
that  the  diagram  shows  a  unidirectional  pathway  leading  from  the  course 
of  action  indicated  in  the  column  heading  to  the  course  of  Action  indi¬ 
cated  in  the  row  heading. 

It  will  be  noted  that  for  each  of  the  13  alternative  return-to- 
base  courses  of  action  shown  in  Figure  2,  two  alternatives  are  indi¬ 
cated  in  the  terminal  area.  These  correspond  to  the  availability  or 
non-availability  of  THACAL  facilities. 

It  is  assumed  that  regardless  of  the  course  of  action  followed 
in  the  return-to-base  phase  the  pilot  will  call  the  tower  for  instruc¬ 
tions  prior  to  landing.  Therefore,  all  courses  of  action  are  shown 
converging  at  this  point.  Fol .  .wing  this,  various  sequences  are  shown 
which  represent  the  alternative  landing  r^'^urses  of  action  possible  de¬ 
pending  upon  tower  instructions,  facilities  available,  weather  and 
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TABLE  II 


Summary  of  Pathways  for  Changing  from 
One  Course  of  Action  to  Another 
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traffic  conditions,  possible  system  malfunctions,  etc.  The  part  of 
the  diagram  representing  the  landing  phase  was  constructed  using  the 
same  procedure  as  was  used  fox'  the  return- to-base  phase.  States  of 
the  system  from  which  successful  touchdown  can  be  made  were  listed 
and  then  the  antecedents  to  these  were  traced  back  to  a  common  origin 
at  the  point  where  instructions  are  received  from  the  tower.  Any  one 
of  the  courses  of  action  that  may  be  used  during  the  landing  phase  may 
be  terminated  under  certain  conditions  if  it  becomes  necessary  for  the 
pilot  to  go  around.  This  alternative  is  not  diagrammed  in  the  interest 
of  simplicity. 

The  structure  of  the  diagram  presents  what  appears  at  first 
glance  to  be  a  rather  complicated  program  for  the  pilot.  This  apparent 
complexity  persists  even  though  a  number  of  simplifications  were  intro¬ 
duced  to  reduce  the  number  of  alternative  courses  of  action  shown  on 
the  diagram.  Careful  study,  however,  reveals  that,  once  a  particular 
course  of  action  is  initiated,  it  will  usually  be  carried  through  to 
completion.  A  change  in  a  course  of  action  will  be  made  only  if  some 
event  occurs  to  reader  this  ongoing  course  of  action  inappropriate. 

The  occurrence  of  such  an  event  is  signalled  to  the  pilot,  and  then 
either  he  makes  the  appropriate  response,  or  the  system  makes  the 
appropriate  response  automatically.  In  instances  where  it  appears 
that  insufficient  information  is  presented  to  him  concerning  the  ap- 
propx'iate  response  to  make,  his  standard  operating  procedure  is  to 
make  the  response  that  will  put  the  system  in  the  next  most  sophi'^ti- 
cated  mod®  of  operation. 
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CLASSIFICATION  OF  POSSIBLE  EVENT  SEQUENCES 


It  should  be  emphasized  that  the  diagrams  shown  in  Figures  1  and 
2  indicate  only  an  extremely  small  fraction  of  sequences  of  events 
that  might  occur  during  the  attack,  return- to-base,  and  landing  phases 
of  the  mission.  This  is  a  result  of  the  procedure  used  in  constructing 
the  diagrams.  It  is  probably  worth  repeating  that  the  various  sequences 
were  constructed  by  starting  with  a  relatively  few  states  of  the  sys¬ 
tem  that  characterize  the  successful  completion  of  a  mission  phase,  and 
then  specifying  the  antecedent  events  in  such  a  way  as  to  show  bow  all 
of  the  sequences  may  originate  from  an  arbitrary  common  point.  Only 
in  a  few  instances  (those  sequences  in  Figure  1  terminating  in  a  cross- 
hatched  rectangle  indicating  that  the  attack  is  broken  off)  is  there 
an  indication  of  a  course  of  action  that  does  not  result  in  successful 
completion  of  a  mission  phase. 

Consideration  of  some  of  the  possible  courses  of  action  that  do 
not  result  in  the  successful  completion  of  a  mission  phase  led  to  the 
development  of  a  general  classification  scheme  for  showing  the  various 
possible  kinds  of  event  sequences  that  might  occur.  These  sequences 
are  defined  by  various  combinations  of  and  relationships  between  the 
following  factors:  (1)  anticipation  of  an  event  and  provision  of  an 
appropriate  response  for  it,  (2)  the  pilot’s  diagnosis  of  the  event, 
and  (3)  the  nature  of  the  pilot’s  response. 

An  event  whose  occurrence  renders  an  ongoing  course  of  action 
inappropriate  for  the  successful  accomplishment  of  a  mission  objec¬ 
tive  may  be  put  into  one  of  three  classes.  One  class  is  composed  of 
all  those  events  that  are  anticipated  and  for  w’'<ch  appropriate  re¬ 
sponses  are  provided  in  the  system  design  and  in  the  pilot’s  training 
program  (Type  I  events).  A  second  class  is  composed  of  events  that 
can  be  anticipated  but  for  which  no  specific  appropriate  responses  are 
provided  (Ty^o  II  events).  The  responses  may  not  be  provided  either 
because  their  cost  is  pr'^'  -bitive  or  because  no  specific  appropriate 
responses  are  known.  A  third  class  is  composed  of  those  events  that 
may  occur  during  a  mission  but  which  are  unknown  to  the  system  design¬ 
ers  and  the  pilot  prior  to  their  occurrence  (Type  III  events).  Since 
these  tents  cannot  be  anticipated  no  specific  appropriate  responses 
for  them  can  be  provided,  either  in  the  design  of  the  system  or  in  the 
pilot’s  training  urogram. 

An  event  sequence  may  start  with  the  occurrence  of  any  one  of 
these  three  types  of  events.  The  outcome  of  a  sequence  depends  upon 
the  pilot’s  diagnosis  of  the  event  and  the  nature  of  his  response. 

When  an  event  from  any  one  of  these  classes  occurs  the  pilot  may  diag¬ 
nose  it  correctly,  or  he  may  diagnose  it  incorrectly,  or  he  may  fail  to 
diagnose  it.  For  a  well  designed  system  and  an  adequately  trained  pilot 
it  may  be  expected  that  most  frequently  the  pilot  will  make  a  correct 
diagnosis  of  Type  1  events.  For  Type  III  events,  on  the  other  hand, 
the  probability  may  be  quite  high  that  the  pilot  will  either  diagnose 
the  event  incorrectly  or  fail  to  diagnose  it. 

Following  the  diagnosis  of  an  event,  regardless  of  whether  it  is 
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correct  or  incorrect,  the  pilot  may  make  a  response  appropriate  to 
the  diagnosis,  or  he  may  make  a  response  that  is  inappropriate  to  the 
diagnosis,  or  he  may  make  no  response.  With  a  well  designed  system 
and  an  adequately  trained  pilot  it  may  be  expected  that  when  a  Type  I 
«^vent  is  correctly  diagnosed,  the  pilot  will  most  frequently  make  the 
appropriate  response.  The  fact  that  no  specific  response  is  provided 
in  the  system  design  or  in  the  pilot's  training  for  Type  II  and  Type 
III  events  does  not  necessarily  mean  that  the  pilot  cannot  make  some 
response  that  would  be  appropriate  for  one  of  these  events.  Although 
the  probability  may  be  quite  low,  it  is  conceivable  that  a  response 
or  combination  of  responses  that  was  provided  for  some  anticipated 
event  may  prove  effective  for  a  Type  II  or  Type  III  event. 

A  diagram  illustrating  the  kinds  of  event  sequences  resulting 
from  this  method  of  classification  is  presented  in  Figure  3.  The  dia¬ 
gram  shows  only  the  sequence  that  may  result  when  a  Type  I  event  occurs. 
The  structure  would  be  identical  for  both  Type  II  and  Type  III.  The 
letters  S  and  U  at  the  bottom  of  the  diagram  indicate  successful  and 
unsuccessful  outcomes,  respectively,  of  the  individual  sequences  of 
events  in  terms  of  the  mission  objectives.  The  various  sequences  have 
also  been  labeled  with  lower  case  letters  (a,  b,  c,  ...,  u.)  in  order 
that  they  may  be  more  easily  identified  in  the  following  discussion. 

The  sequences  of  events  considered  above  in  the  analysis  of  the 
various  phases  of  the  F-106/MA-1  mission  were  a-type  sequences.  The 
various  courses  of  action  that  v/ere  diagrammed~represented  appropriate 
responses  made  to  correctly  diagnosed  events  that  were  anticipated  in 
the  design  of  the  system  and  in  the  pilot’s  training  program. 

Let  us  now  consider  some  of  the  other  possible  kinds  of  event 
sequences  originating  with  Type  I  events.  It  is  possible  for  the 
pilot  to  diagnose  the  event  correctly  but  to  make  an  inappropriate 
response.  This  might  come  about  as  a  result  of  poor  coding  of  controls 
and  control  setting  positions  or  through  inadequate  training  in  the 
operation  of  the  system.  In  many  cases  making  an  inappropriate  re¬ 
sponse  would  be  disastrous  (sequence  d ) .  On  the  other  hand,  it  is 
possible  that  the  pilot  may  be  informied  of  the  incorrectness  of  the 
response  by  means  of  feedback  from  the  system  or  from  the  performance 
of  the  response  itself.  If  such  is  the  case,  the  response  may  be 
corrected  either  in  time  to  result  in  a  successful  outcome  (sequence 
b)  or  too  late  to  result  in  a  successful  outcome  (sequence  c) .  It  is 
also  possible  for  an  event  to  be  diagnosed  correctly  but  for  no  re¬ 
sponse  to  be  made.  The  outcome  here  would  of  course  be  unsuccessful 
(sequence  g) ,  In  the  case  where  "no  response  made"  refers  only  to  a 
delay  in  making  a  response,  there  are  two  possible  event  sequences. 

One  of  these  (sequence  f)  repi'esents  the  case  where  the  response  is 
made  too  late  to  be  effective.  The  other  sequence  (e)  places  the  pilot 
back  at  either  1  or  2  on  the  diagram.  ~ 

If  the  pilot  diagnoses  a  Type  1  event  incorrectly  and  makes  a 
response  appropriate  tc  his  diagnosis,  the  outcome  may  be  unsuccessful 
(sequence  h) .  There  is  another  possibility  here,  however.  Since  the 
response  is”  inappropriate  for  the  actual  event,  if  the  pilot's  dis¬ 
plays  provide  good  feedback  information,  he  will  make  a  new  diagnosis 
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of  th«  state  of  the  system.  This  may  occur  too  late  to  be  of  any 
value  (sequence  k) ,  or  it  may  start  a  new  sequence  at  either  A  or  B 
on  the  diagram  (sequence  ^)  ,  When  an  event  is  diagnosed  incorrectly 
add  a  I’esponse  inappropriate  to  the  diagnosis  is  made  there  is  some' 
low  probability  that  the  response  will  be  appropriate  to  the  actual 
event  (sequence  ^) .  The  more  likely  outcome,  however,  of  an  inap- 
prop  iate  response  made  to  an  incorrectly  diagnosed  event  is  failure 
(sequence  m) ,  Once  again  there  is  the  possibility  of  a  new  diagnosis 
which  may  occur  too  late  (sequence  o) ,  or  may  start  a  new  sequence  at 
either  A  or  B  (sequence  n) .  In  the  case  where  an  event  is  diagnosed 
incorrectly  and  no  response  is  made  the  outcome  may  be  unsuccessful 
(sequence  r ) .  If  some  response  is  made  eventually  it  may  be  too  late 
to  be  effective  (sequence  q)or  it  may  start  a  new  sequence  at  either 
4  or  5  on  the  diagram  (sequence  p) . 

If  a  Type  I  event  is  not  diagnosed  it  is  reasonable  to  expect 
that  no  response  will  be  made.  The  outcome  of  this,  of  course,  is 
unsuccessful  (sequence  s) .  Once  again,  however,  there  is  the  pos¬ 
sibility  that  a  diagnosrs  will  eventually  be  made.  If  it  is  made, 
the  sequence  either  starts  over  at  A  or  B  on  the  diagram  (sequence  t) 
or  if  the  diagnosis  is  made  too  late  the  outcome  is  unsuccessful  ~ 

(sequence  u) , 

The  question  may  be  asked,  "If  the  kinds  of  possible  event 
sequences  are  the  same  for  Type  II  and  Type  III  events  as  for  Type  I 
events,  why  make  a  distinction  among  these  three  types  of  events?" 

The  answer  to  this  question  is  that  the  probabilities  of  occurrence 
of  the  individual  sequences  are  different  for  the  three  types  of  events. 
For  example,  consider  sequence  a.  While  it  may  be  the  most  probable 
of  all  the  sequences  when  a  Type  I  event  occurs,  its  probability  of 
occurrence  may  be  quite  low  for  a  Type  II  event,  and  perhap.s  it  may  be 
the  least  probable  of  all  the  sequences  when  a  Type  III  event  occurs. 

In  view  of  the  fact  that  the  manner  of  conceptu?  1. iz.ing  decision 
making  was  arrived  at  through  the  study  of  examples  of  only  one  of  the 
several  possible  kinds  of  event  sequences,  the  generality  of  this  con¬ 
ceptualisation  was  subjected  to  examination.  The  conclusion  was  reached 
that  no  modification  of  the  concept  was  required  in  order  for  it  to  be 
generally  applicable. 

The  decision  problem  for  the  pilot  is  the  same  regardless  of 
which  type  of  event  occurs.  While  it  is  true  that  the  pilot  niay  have 
had  no  specific  training  in  the  diagnosis  of  Type  III  events,  for  ex¬ 
ample,  the  problem  confronting  him  when  one  of  these  events  occurs  is 
the  same  as  the  problera  facing  him  when  a  known  event  (Type  I  or  Type 
II  )  occurs.  In  all  cases  the  decision  required  is  a  determination 
of  the  state  of  the  system.  .'Ithough  the  probability  of  a  correct 
diagnosis  may  be  lower  for  Type  III  events,  the  essential  characteris¬ 
tics  of  the  decision  process  are  the  same.  No  matter  how  novel  the 
event,  if  it  is  sensed  at  all,  it  presumably  has  some  features  in  com¬ 
mon  with  kno«n  events.  The  response  that  has  the  highest  probability 
of  being  made  is  a  response  appropriate  to  the  known  event  most  simi¬ 
lar  to  the  diagnosed  event. 
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The  analysis  of  the  various  possible  kinds  of  event  sequences 
also  illustrates  some  of  the  advantages  of  a  manned  system  over  a 
completely  automatic  system.  One  of  these  advantages  may  be  realized 
in  the  case  of  Type  III  events.  Since  these  events  are  unknown  prior 
to  their  occurrence,  they  cannot  be  anticipated  and  an  automatic  sys¬ 
tem  cannot  be  specifically  designed  to  cope  with  them.  While  the 
probability  of  a  man  correctly  diagnosing  and  appropriately  respond¬ 
ing  to  one  of  these  events  may  not  be  very  high,  it  is,  nevertheless, 
much  higher  than  it  would  be  for  an  automatic  system. 

An  advantage  may  also  be  realized  from  having  a  man  in  the  sys¬ 
tem  in  the  case  of  Type  II  events.  These  events  are  those  that  can  be 
anticipated  but  for  which  no  specific  appropriate  responses  are  pro¬ 
vided.  Although  no  specific  responses  have  been  provided,  the  man  has 
a  large  program  of  responses  appropriate  for  other  events.  There  is 
some  probability,  however  low,  that  a  response  or  responses  provided 
for  Type  I  events  may  prove  effective  against  a  Type  II  event.  The 
required  complexity  of  automatic  airborne  equipment  designed  to  imi¬ 
tate  the  man  in  this  respect  is  presently  prohibitive  in  terms  of  size, 
weight,  and  dollar  cost. 

There  is  another  point  to  be  noted  in  reference  to  the  advantages 
of  a  manned  system  over  an  automatic  system  when  both  Type  II  and  Type 
III  events  are  considered.  The  response  made  by  the  man  to  one  of  these 
events  may  not  be  effective  in  coping  with  the  event.  It  may,  however, 
alter  the  state  of  the  system  so  that  it  is  then  characterized  by  the 
existence  of  a  Type  I  event,  which  may  have  a  high  probability  of  being 
correctly  diagnosed  and  for  which  an  appropriate  response  has  beeti  pro¬ 
vided  . 
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SYSTEMS  DESCRIPTION 


It  is  clear  frcm  the  foregoing  analysis  that  the  operator  of 
a  weapon  system  is  not  so  concerned  with  choosing  among  alternative 
courses  of  action  as  he  is  with  deciding  what  the  momentary  state  of 
the  system  is.  Once  a  diagnosis  is  made,  the  proper  course  of  action 
is  specified  by  operating  rules  implicit  in  the  design  of  the  system. 

In  other  words,  the  system  possesses  a  built-in  operating  logic  of  the 
form;  if  event  E  occurs  then  make  response  R.  The  set  of  all  such 
events  together  with  their  responses  constitutes  what  might  be  called 
the  program  of  the  system. 

All  weapon  systems  have  programs  of  this  kind.  Provision  is 
made  for  detecting  the  occurrence  of  some  restricted  set  of  events, or 

classes  of  events,  [Ei,  E2, - Ei, - Ejjj]  ,  where  each  E  may 

be  either  a  single  discrete  event  such  as  the  occurrence  or  non-occur¬ 
rence  of  some  particular  kind  of  countermeasure  or  else  a  class  of 
events  such  as  the  set  of  all  deviations  from  desired  course  greater 
than  some  minimvm.  The  events  provided  for  in  a  particular  program 
are  selected  from  a  much  larger  set,  all  members  of  which  have  the 
property  that  their  occurrence  would  prevent  the  successful  accomplish¬ 
ment  of  the  mission  or  at  least  would  lower  the  probability  of  success 
if  nothing  were  done  in  counteraction. 

The  response  provided  by  the  system  is  designed  to  counteract  the 
effects  of  the  event  that  occurs;  i.e.,  it  is  designed  to  restore  the 
probability  of  completing  a  successful  mission.  It  is  therefore  called 
an  adaptive  response. 

In  an  unmanned  system  the  program  is,  of  course,  mechanized  in 
the  system.  In  a  manned  system  the  program  is  usually  only  partly 
mechanized  with  the  man  providing  the  remaining  detection  and  logic 
functions.  A  program  may  be  large  or  small  depending  upon  the  number 
of  events  the  designer  feels  must  be  considered  in  order  to  have  a 
successf\il  system  and  vipon  the  cost  of  providing  for  them  in  the  pro¬ 
gram  . 


When  choosing  .a  program  for  a  system  the  designer  attempts  to 
find  a  satisfactory  tradeoff  among  a  number  of  quantities.  The  first 
quantity  is  the  a  priori  probability  that  each  particular  event  will 
occur  during  a  mission .  Some  events  are  much  more  likel>  than  otiiers. 
Secondly,  there  is  associated  with  each  event  some  measure  of  its 
threat  to  the  successful  accomplishment  of  the  mission.  These,  too, 
covei'  a  cOnsiclci’able  range  of  values.  Frequently  occurring  events 
that  are  effective  in  disrupting  the  mission  must  obviously  be  com¬ 
pensated  for  in  any  practical  system.  For  example,  most  missiles  have 
gidance  systems  of  one  kind  or  another  in  order  to  compensate  for  per- 
turb.ations  arising  from  initial  aiming  errors,  rough  air,  instability, 
and  so  on.  On  the  other  hand,  the  program  might  contain  no  provision 
for  even  a  completely  disastrous  event  if  it  were  felt  that  the  proba¬ 
bility  of  its  occurrence  were  very  lou . 

A  third  quantity  to  be  considered  when  choosing  a  program  is  i he 
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cost  of  providing  for  each  event  in  the  program.  The  cost  may  be 
weight,  space  and/or  dollars.  The  cost  comes  from  at  least  two 
sources,  (1)  the  cost  of  equipment  for  sensing  or  detecting  the 
occurrence  of  the  event,  and  (2)  the  cost  of  providing  an  appropriate 
response  for  the  system  to  make  should  the  event  occur. 

A  fourth  quantity  that  enters  into  the  tradeoff  is  the  proba¬ 
bility  that  the  response  will  be  effective  in  counteracting  the  effects 
of  the  event.  For  example,  most  airplanes  carry  fire  extinguishing 
systems  to  be  used  in  the  event  of  a  fire.  This  equipment  is  not  al¬ 
ways  successful  in  putting  out  a  fire,  and  yet  it  must  have  a  suffi¬ 
ciently  high  probability  of  success  in  order  to  be  worth  carrying. 

When  selecting  a  program  for  some  weapon  system  the  designer  is, 
in  a  sense,  actually  making  tactical  decisions  for  the  system  and  the 
process  is  theoretically  susceptible,  at  least,  to  a  description  in 
terms  of  decision  theory.  It  is  possible  in  principle,  for  example, 
to  compute  the  expected  value  of  a  system  having  a  particular  program. 
This  could  be  done  again  for  the  same  system  with  different  programs, 
and  that  program  yielding  the  highest  expected  value  could  then  be 
chosen.  In  a  sense  this  is  what  is  done  during  the  planning  stage  for 
a  weapon  system,  even  though  it  may  not  be  practical  to  conduct  the 
formal  exercise. 

In  the  planning  phase  the  mission  for  the  weapon  is  defined. 

This  is  a  statement  of  what  the  weapon  must  do.  For  example,  it  must 
fly  1000  miles,  drop  a  bomb  weighing  a  certain  amount  with  an  accuracy 
of  such  an  amount  and  return  to  base  under  all  weather  conditions. 

Next,  the  designer  tries  to  decide  what  he  must  build  in  order  to  be 
able  to  do  the  missions  at  all.  .At  this  point  he  may  consider  alter¬ 
native  kinds  of  systems,  i.e.,  a  missile,  a  manned  aircraft,  and  so  on. 
These  decisions  involve  program  making,  because,  although  each  basic 
type  of  weapon  might  be  capable  of  performing  an  uneventful  mission, 
that  type  will  be  chosen  that  offers  the  highest  expected  value  in  the 
face  of  the  major  events  that  are  expected  to  occur  during  a  typical 
mission . 

When  the  basic  type  of  weapon  is  chosen,  then  detailed  program 
making  becomes  one  of  the  major  concerns  of  the  system  designers. 

Since  the  interest  of  this  project  lies  with  manned  systems  it  will  be 
assumed  that  a  manned  system  has  been  chosen.  The  fact  tlial  a  manned 
system  was  chosen  probably  indicates  that  the  program  deemed  necessarj 
for  the  successful  employment  of  the  weapon  is  a  large  one  and  fnat  the 
occurrence  of  some  of  the  events  in  it  cannot  easily  be  detected  avito- 
matical ly . 

But  beca'.  se  it  is  a  manned  sjstem  the  program  i.s  obliged  to  as¬ 
sume  some  special  characteristics,  one  of  which,  tends  to  complicate 
the  system  while  a  second  tends  to  simplify  it. 

The  first  characteristic  i.s  concerned  with  the  redundancy  that 
must  be  built  into  the  system  bec.ausc  thei'e  is  a  man  in  it.  Thai  is  to 
say  that  a  large  subset  of  the  events  provide'd  for  in  the  program  ar'' 
events  of  evTuipment  m.a  1  f  unc  t  ion  .  Tlius  ,  if  event  ii-^,  a  fail.ire,  occurs, 
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then  response  ,  an  alternate  mode  of  operation,  is  provided,  in 
order,  at  least,  to  return  the  aircraft  safely  to  base  if  not  to  con¬ 
tinue  its  mission.  These  event-response  pairs  range  in  kind  from  di¬ 
rect  counteractions  to  the  event,  as  in  the  case  of  fire  extinguishing, 
to  the  selection  of  alternate  modes  of  performing  the  mission,  as  might 
be  the  case  when  an  optical  mode  is  used  as  backup  for  a  radar.  It 
should  be  noted  in  contrast  that  missiles  are  not  normally  built  this 
way  now,  and  therefore  an  equipment  failure  in  a  missile  has  a  high 
probability  of  resulting  in  an  unsuccessful  mission. 


The  switching  necessary  to  make  the  response,  R,  is  most  fre¬ 
quently  an  operator  function  in  manned  systems.  This  is  true  even 
though  the  switching  could  be  performed  automatically.  This  leads  to 
the  second  characteristic  of  manned  systems,  namely,  that  the  man  may 
be  required  in  the  system  only  to  accomplish  a  small  part  of  the  pro¬ 
gram  but  that,  since  he  is  there  for  this  purpose,  he  is  called  on  for 
other  parts  of  the  program  as  well. 

Consider  an  e’-trerae  and  fictional  example.  Suppose  that  the 
primary  reasons  for  employing  a  manned  interceptor  are  (1)  to  make  a 
positive  visual  identification  of  a  target  before  firing,  (2)  to  achieve 
radar  lockon  to  a  target  in  the  presence  of  ground  clutter,  and  (3)  to 
act  against  certain  countermeasures  that  the  enemy  might  employ.  Now 
it  would  be  possible  to  build  an  interceptor  system  that  would  carry 
the  man  as  a  passenger  except  for  the  performance  of  those  three  func¬ 
tions.  However,  no  one  at  this  time  would  con.sider  it  worth  while  to 
do  so.  Since  (.he  man  is  already  there  at  great  expense  he  might  as 
well  be  used  up  to  the  efficient  •  limit  of  his  capability.  Hence,  he  is 
given  responsibility  for  much  more  of  the  program  than  the  three  events 
for  which  he  is  primarily  responsible. 

Manned  systems  possess  some  other  attributes  of  interest  that  may 
or  may  not  contribute  towards  the  success  of  a  mission.  One  of  these 
is  flexibility.  Let  some  event,  Ep; ,  occur  during  a  mission  where  Ej^ 
was  not  originally  provided  for  in  the  progiuam  of  the  system.  A  manned 
system  compared  with  an  unmanned  system  stands  some  chance  of  lespond- 
ing  correctly  to  this  event  if  it  can  be  detected.  Crudely  stated,  the 
man  is  able  by  inductive  reasoning  to  classify  the  new  event  as  being 
similar  to  some  event  already  included  in  the  program,  and  the  response 
designated  for  the  included  event  may  be  appropriate  to  the  unexpected 
event . 


4  second  way  in  which  flexibility  may  be 
ing  of  the  occurrence  cf  particular  events.  No 
larly  an  unmanned  system,  accomplishes  the  sens 
occurrence  of  each  event  in  the  program  by  niean 
inputs  provided  for  the  purpose.  For  example, 
craft  On  final  approach  may  be  detected  by  mean 
ixer  '’ud  glide  path  signals.  Should  these  sign 
tern  f  c  whatever  reason  then  the  event  goes  und 
btlity  of  mission  success  is  decreased.  .Manned 
tically  flexible  in  this  respect  in  that  altern 
tion  can  often  be  improvised.  This  is  particul 
events.  For  e.Kample,  suppose  that  some  manned 


achieved  is  in  the  sens- 
rmally  a  system,  part  idl¬ 
ing  or  detection  of  the 
S  of  a  particular  set  of 
the  position  of  air- 
s  of  a  set  of  ILS  local - 
als  be  denied  to  the  sy.s- 
etected  and  the  proba- 
s  vs  terns  are  chai'ac  ter  is- 
ative  methods  of  detec- 
arly  true  of  coraplex 
aircraft  were  inlended 
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to  make  instrument  approaches  on  low  frequency  radio  ranges.  The  air¬ 
craft  attempts  to  make  such  an  approach  but  finds  that  the  low  fre¬ 
quency  receiver  is  out  of  comiission.  However,  automatic  direction 
finding  equipment  is  available  for  detecting  the  bearing  of  the  range 
station.  It  is  now  possible  to  fly  exactly  the  same  approach  using 
visual  ADF  indications  and  compass  headings  as  would  have  been  flown 
with  the  low  frequency  range  audio  signals,  even  though  this  procedure 
was  not  intended  to  be  used  when  the  system  was  designed.  Flexibility 
of  this  type  is  not  commonly  found  in  unmanned  systems. 

The  above  discussion  is  intended  to  show  that  as  a  consequence 
of  the  way  in  which  systems  are  designed,  it  is  possible  to  anticipate 
most  of  the  decisions  the  operator  of  the  system  will  be  expected  to 
make.  These  are  of  the  diagnostic  type;  the  operator  is  trying  to  de¬ 
cide  what  the  state  of  his  systen  is  or  will  be.  Many  of  these  deci¬ 
sions  are  "easy”  to  make,  the  information  displays  showing  clearly 
what  is  happening.  But  many  of  the  decisions  are  difficult.  The  in¬ 
formation  is  noisy  or  indirect  or  it  may  (common! vl  be  displayed  in 
bits  and  pieces  through  a  number  of  instruments,  ,  .airing  integration 
by  the  operator. 

How  the  human  makes  decisions  of  this  kind  has  recently  become 
a  subject  of  great  interest.  The  interest  has  been  sparked  by  the  de¬ 
velopment  of  statistical  decision  theory  and  game  theory  in  mathematics 
and  economics.  Psychologists  have  taken  up  the  new  models  and  are  ex¬ 
perimenting  to  determine  how  well  they  fit  in  the  case  of  real  h'  ^n 
decision  making  under  laboratory  conditions. 

Immediate  application  of  the  models  to  the  case  of  pilot  decision 
making  with  the  expectation  of  useful  quantitative  resvsl  Is  is  premature 
at  this  time.  Nevertheless,  it  can  be  assumed  with  some  confidence  that 
in  the  future  the  systematic  thinking  about  human  decision  making  in 
real  life  situations  will  derive  from  this  work.  Therefore,  it  is  of 
interest  to  review  briefly  the  current  thought  on  the  problem. 
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APPLICABILITY  OF  VARIOUS  DECISION  THEORIES  TO  THE  AIRBORNE 

DECISION-MAKING  PROBLEM 


This  section  very  briefly  outlines  in  general  terms  some  of  the 
sajor  theories  concerned  with  decision  making  and  discusses  the  pos¬ 
sible  applicability  of  the  theories  to  the  airborne  decision-making 
problem,  A  detailed  treatment  of  the  historical  development  of  the 
theories s  some  of  the  special  problems  associated  with  each  theory, 
and  attempts  to  test  various  hypotheses  may  ne  found  in  a  review  arti¬ 
cle  by  Edwards  (77) .  More  comprehensive  and  rigorous  statements  of 
the  various  theories  may  be  found  in  the  following  references; 

Girshick  (78),  McKinsey  (86),  Chernoff  (30),  Thrall,  Coombs,  and  Davis 
(51),  Wald  (52)  and  VonNeuraann  and  Morgenstern  (88), 


TBEOEY  OF  RISKLESS  CHOICES 

The  theory  of  riskless  choices  is  applicable  only  to  the  choices 
made  by  what  is  termed  an  ’’economic  man.  "  It  is  assumed  that  economic 
man  has  complete  information.  He  knows  all  of  tiie  alternative  courses 
of  action  available,  and  he  knows  what  the  result  or  outcome  of  each 
of  the  courses  of  action  will  be.  Economic  man  is  assumed  to  be  in¬ 
finitely  sensitive,  and  the  alternatives  available  to  him  are  assumed 
to  be  continuous,  infinitely  divisible  functions.  It  is  further  as- 
svtmed  that  economic  man  is  rational.  Rationality  implies  that  he  can 
weakly  order  the  utilities  of  the  outcomes  of  the  alternative  courses 
of  action,  and  that  he  makes  his  choice  so  as  to  maximize  utility. 
According  to  this  theory,  economic  man  maximizes  utility  by  choosing 
the  alternative  course  of  action  having  the  outcome  with  the  highest 
utility , 

This  theory  is  not  applicable  to  tbe  practical  airborne  decision 
making  problem.  One  may  be  willing  tentatively  to  make  the  assumption 
that  the  pilot  meets  the  requirements  of  rationality.  The  assumptions 
concerning  infinite  sensitivity  may  be  abandoned  (as  they  have  in  some 
versions  of  the  theory).  The  assumption  that  the  pilot  has  complete 
information,  however,  cannot  be  made.  Even  though  he  may  know  in  most 
cases  what  the  available  alternative  courses  of  action  are,  he  never 
knows  with  complete  certainty  what  the  outcome  of  any  given  course  of 
action  will  be. 

THEORY  OF  RISKY  CHOICES 

The  theory  of  risky  choices  takes  into  account  the  fact  that  an 
individual  who  must  make* a  choice  never  has  complete  information  in 
advance  about  the  outcome  of  his  choice.  The  choice  situation  is  chai’- 
acterized  by  the  existence  of  two  or  more  alternative  coxirses  of  action, 
for  each  of  which  there  are  two  or  more  possible  outcomes.  The  theory 
requires  that  the  individual  making  the  choice  must  know  the  alternative 
courses  of  action.  He  must  also  know  the  possible  outcomes  for  each  cf 
the  courses  of  action  although  he  does  not  know  which  outcome  will 
actually  occur. 
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It  is  assumed  that  the  individual  can  assign  a  probability  of 
occurrence  and  a  utility  to  each  possible  outcome.  According  to  the 
theory  the  individual  chooses  his  course  of  action  in  such  a  way  as 
to  maximize  the  expected  utility.  The  expected  utility  of  a  given 
course  of  action  is  calculated  by  multiplying  the  utility  of  each  of 
the  possible  outcomes  by  its  probability  of  occurrence  and  summing 
these  products  across  all  of  the  possible  outcomes. 

This  theory  is  similar  in  some  respects  to  the  version  of  de¬ 
cision  theory  derived  from  detection  ?>ieory  that  is  discussed  else¬ 
where  in  this  paper.  Both  theories  take  into  account  the  probabilis¬ 
tic  nature  of  the  individual's  knowledge  about  events,  and  both  con¬ 
sider  the  costs  and  values  or  utilities  of  possible  outcomes.  A 
major  difference  between  the  two  theories  is  the  aspect  of  the  indi¬ 
vidual’s  behavior  that  is  focused  upon.  The  aspect  of  choosing  em¬ 
phasized  by  the  theory  of  risky  choices  is  the  weighing  of  alternative 
courses  of  action  to  determine  which  should  be  taken.  Detection 
theory,  on  the  other  hand,  emphasizes  the  discrimination  aspect  of 
choosing  and  considers  the  course  of  action  to  be  determined  by  the 
results  of  the  discrimination.  Thus,  of  the  two  theories,  detection 
theory  seems  to  be  the  more  applicable  to  decision  making  as  it  is 
conceptualized  in  this  report.  Even  when  decision  making  is  concep¬ 
tualized  in  terms  of  selecting  an  alternative  there  are  serious  dif¬ 
ficulties  in  applying  the  theory  of  risky  choices  to  practical  situ¬ 
ations.  These  difficulties  are  concerned  with  the  assignment  of  prob¬ 
abilities  and  utilities  to  the  various  possible  outcomes.  Until  con¬ 
siderably  more  work  is  done  on  the  problems  of  scaling  utilities,  re¬ 
lating  subjective  estimates  of  probabilities  to  objective  probabilities, 
etc.,  there  can  be  no  adequate  test  of  the  applicability  of  this  model 
to  airborne  decision  making. 


G/'ME  THEORY 

Game  theory  provides,  for  cei'tain  well-defined  situations,  a  set 
of  rules  for  selecting  a  course  of  action  ftom  among  the  alternatives 
available.  The  theory  is  not  concerned  with  the  problem  of  how  an  in¬ 
dividual  actually  selects  a  course  of  action  in  a  situation  to  which 
the  theory  is  applicable,  but  rather  with  the  problem  of  determining 
what  course  of  action  should  be  selected.  The  rules  for  selecting  a 
course  of  action  are  aimed  at  minimizing  the  maximum  loss. 

For  a  zero-sum,  two-person,  finite  game,  which  incidentally,  the 
pilot  seldom,  if  ever,  participates  in,  the  requirements  are  as  follow: 
(1)  The  pilot  must  know  all  the  alternative  courses  of  action  available 
to  him  and  all  of  the  alternative  courses  of  action  available  to  his 
opponent.  (2)  He  must  be  able  to  assign  a  payoff  value  to  each  combi¬ 
nation  of  his  own  possible  courses  of  action  with  each  of  his  opponent's 
courses  of  action.  (3)  Ke  must  then  know  and  use  the  rules  for  selec¬ 
ting  the  course  of  action  that  will  minimize  his  maximum  loss  (or  maxi¬ 
mize  his  minimum  gain)  . 

Ouite  apart  from  the  fact  that  the  theory  developed  for  zero-su.ra, 
two-person  games  is  seldom  applicable  to  the  airborne  decision-making 


situation  there  are  other  reasons  for  believing  that  an  atter-pt  to 
directly  apply  the  theory  would  not  be  particularly  profitable.  While 
the  pilot  may  know,  in  some  cases,  the  alternative  courses  of  action 
available  to  him,  he  does  not  necessarily  know  a31  of  the  alternative 
courses  of  action  available  to  his  opponent.  Furthermore,  the  assign¬ 
ment  of  payoffs  to  the  various  combinations  of  each  of  his  own  courses 
of  action  with  each  of  the  opponent's  courses  of  action  represents  a 
pi’Oblem  for  which  there  dees  not  seem  to  be  an  adequate  solution  at 
present . 

It  would  seem  that  game  theory  may  be  most  fruitfully  applied 
in  studies  preliminary  to  the  design  of  weapons  system  rather  than  to 
the  decision-making  problems  faced  by  the  pilot.  That  is  to  say,  it 
appears  useful  in  building  a  program  for  a  system  but  not  in  the  oper¬ 
ation  of  the  system  as  described  here. 


A  DECISION-MAKING  THEORY  OF  DETECTION 

The  last  theory  to  be  considered  here  is  the  theory  of  visual 
detection  developed  by  Tanner,  Swets,  et  al . ,  (14),  from  the  earlier 
theory  of  signal  detectability  of  Peterson  and  Birdsall  (10) .  This 
theory  has  an  intuitive  appeal  for  application  to  the  problem  of  pilot 
decision  making  even  though  it  is  true  that  no  quantitative  applica¬ 
tion  is  possible  now.  The  justification  for  including  it  here  is  that 
it  may  be  made  to  serve  what  Lazarsfeld  (32)  calls  the  organizing  or 
linguistic  as  opposed  to  the  predictive  function  of  a  moH^'H  TKat  is 
to  say  that  the  concepts  of  the  model  may  be  useful  in  organizing  a 
diverse  set  of  observations  at  the  linguistic  level  even  though  pre¬ 
cise  predictions  of  behavior  cannot  be  made  because  quantitative  data 
are  not  available. 

Although  the  Tanner  model  began  as  a  theory  of  visual  detection, 
it  has  since  been  extended  to  include  auditory  detection  and,  more  re- 
ce.  -tly,  8  theory  of  recognition,  A  general  theory  of  perception  is 
the  announced  goal.  The  theory  is  mathematical.  The  experiments  that 
Tanner  has  done  to  test  the  application  of  the  theory  depend  upon  q:'an- 
titatlve  prediction  of  behavior  and  therefore  employ  restricted  and 
well-controlled  experimental  tasks  and  situations  to  an  extent  not  fea¬ 
sible  when  studying  pilot  decision  making. 

At  the  risk  of  doing  great  violence  to  the  theory  and  by  deliber¬ 
ately  ignoring  its  finer  points,  an  attempt  will  be  made  to  abstract 
from  it  what  appears  to  be  pertinent  to  the  present  study. 

Originally  the  theory  dealt  with  a  situation  in  which  the  subject 
acts  as  an  observer  in  a  p.sychophysical  experiment.  The  observer's 
task,  for  example,  could  be  to  look  at  a  display  which  might  be  an  even 
ly  illuminated  sio'-face  and,  at  a  given  time,  to  decide  if  a  signal-a 
smaller  test  patch  of  near  threshold  illumination  within  the  field—  was 
or  wsis  not  presented  by  the  experimenter.  The  subject  knew  In  advance 
what  the  signals  look  like  and  the  time  for  making  the  observstior  was 
defined  for  hits. 
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Unlike  conventional  psychophysical  theory,  Tanner  postulates 
that  the  observer  is  really  a  noisy  receiver  of  signals  and  that  the 
decision  about  the  presence  or  absence  of  the  signal  is  based  upon 
noisy  inf oi'mation,  the  noise  coming  from  random  neural  activity  in 
the  sensoz’y  system.  Hence  the  decision-maker  (to  be  thought  of  as  a 
separate  "compartment"  of  the  observer)  has  the  task  of  testing  statis¬ 
tical  hypotheses  since  an  observation  that  appears  to  have  come  from 
a  signal  could  actually  have  come  from  noise  alone.  Such  an  event, 
however,  can  occur  only  with  some  probability.  Hence,  two  pr...  ^ability 
density  functions  can  be  defined.  One,  f^Cx) ,  describes  the  probabili¬ 
ty  density  of  the  observation,  x,  in  a  noise  distribution.  The  other, 
fSN(x)>  describes  the  probability  density  of  x  in  a  signal-plus-noise 
distribution.  Upon  making  a  particular  observation  the  subject  must 
"decide"  whether  what  he  saw  was  signal  or  noise.  Clearly  if  the  prob¬ 
ability  were  high  that  the  observation,  x,  came  from  the  signal-plus- 
noise  distribution,  compared  with  the  probability  that  it  came  from  the 
noise  distribution  alone,  then  the  best  bet  would  be  to  call  the  obser¬ 
vation  a  signal.  If  the  converse  were  true  then  the  best  bet  would  be 
to  decide  that  a  signal  was  not  present  and  that  noise  alone  was  ob¬ 
served.  The  theory  of  signal  detectability  describes  how  an  "ideal" 
receiver  makes  this  decision  in  all  cases.  Tanner  has  essentially 
adopted  this  model  and  applied  it  to  the  human  observer. 

While  it  is  intuitively  clear  that  the  observation  should  be  called 
a  signal  if  very  much  greater  than  f  jj(x)  ,  in  m.nny  cases  vi 

interest  the  probabilities  are  not  so  "idely  separated,  yet  a  decision 
has  to  be  laa  ;e.  The  theory  of  signkj.  detectability  shows  how  this  can 
be  done  according  to  a  number  of  related  criteria,  each  of  which  maxi¬ 
mizes  some  characteristic  of  the  outcome. 

A  useful  relationship  between  fgjqCx)  and  fjj(,:c)  is  defined  as  the 
likelihood  ratio  J(x)  =  f  sj^(x) /f  m(x)  . The  problem  is  to  find  the  small¬ 
est  likelihood  ratio  that  one  will  accept  and  still  call  the  observation 
a  signal.  Then  all  observations  with  likelihood  greater  than  this  will 
be  called  signals,  and  all  observations  with  likelihood  less  than  this 
will  be  called  noise,  or  signal  not  present.  This  is  the  cutoff  point. 
The  theory  shows  a  number  of  different  ways  of  setting  the  cutoff,  all 
of  v;hich  meet  the  mathematical  requirements  of  being  an  optimum  cri¬ 
terion  . 

The  criterion  of  interest  here  is  called  the  expected  value  cri¬ 
terion  in  which 

r^utoff  =  fl  =  P(N)  (Vr+%) 

^  ■  TVi^MT  * 

P(N)  and  P(SN)  are  the  a  priori  probabilities  of  no  signal  and  signal 
respectively.  It  is  intuTtiveTy  reasonable  that  0  should  be  determined 
in  part  by  the  ratio  of  these  probabilities.  Suppose  that  each  of  these 
probabilities  were  0.5,  i.e.,  50%  of  the  time  signals  are  presented  and 
50%  of  the  time  they  are  not,  then  ;9  -  1 .  In  turn  this  imp]  ie.s  that 
other  things  being  equal  for  an  observation  to  be  called  a  signal  xt 
should  have  ^zt  least  a  50%  chance  of  coming  fi'om  the  sigiia.l-plus-noi.se 
d.istr ibution  and  that  all  those  with  greater  than  50%  chance  should  be 


called  signals  i.e.  A(x)  ^  fsN(x)/fN(x)  ^  1.  Suppose,  in  another 
case,  that  P(SN)  ~  0.01  and  P(N)  =  0.99.  Now  the  minimum  likelihood 
ratio  should  be  set  much  higher  in  order  to  avoid  a  very  large  number 
of  false  alarms.  It  can  be  shown  that  if  ^  were  defined  only  as  /S  = 
P(N)/P(SN)  then  the  observer  using  this  criterion  would  act  to  mini¬ 
mize  his  total  error. 


Now  consider  the  second  term  in  the  expected  value  criterion 

Here 

-  value  of  a  detection  (correct  call  "signal  present") 

Vjj  = 'value  of  a  rejection  (correct  call  "no  signal  present") 

Kjj  =  cost  of  a  miss  (signal  present  but  not  called) 

%  =  cost  of  a  false  alarm  (signal  absent  but  called  as 

present) . 

Again  consider  the  case  where  the  a  priori  probabilities  of  signal 
and  noise  are  equal,  but  in  this  instance  let  the  value  of  a  correct 
detection  far  outweigh  the  cost  of  a  false  alarm.  Now  it  might  be 
better  to  lower  /9  somewhat  in  order  to  catch  a  higher  percentage  of 
the  true  signals  even  at  the  cost  of  many  false  alarms.  On  the  other 
hand  if  false  alarms  were  very  expensive  compared  to  the  value  of  cor¬ 
rect  detections  then  $  should  reasonably  be  raised.  Thus  this  crite 
ion  can  be  said  to  maximize  the  total  expected  value  of  the  results 
the  decision  process. 


Actually,  that  ^  is  an  expected  value  solution  can  be  seen  f 


roin 


the 

following: 

Let  A 

represent  the  set  of  outcomes  in  which  the  ob¬ 

server  says  "a 

signal 

is  present"  and  CA  the  set  of  outcomes  in  which 

he 

says  "a  signal  is 

not  present," 

then  four  outcomes  are  possible: 

SN-A 

- 

a  signal  is 

present  and  is  called  present. 

N.CA 

:: 

a  signal  is 

absent  and  is  called  absent. 

SN.CA 

r 

a  signal  is 

present  and  is  called  absent. 

If 

N*  A 

- 

a  signal  is 

absent  and  is  called  present. 

value  of  the 

correct  response  SN*A 

’'^N«CA 

value  of  the 

correct  response  N-CA 

^SN'CA 

- 

cost  of  the 

error  SN*CA 

A 

cc  t  of  the 

error  N'A 

and  if  P(SN*A),  P(N-‘CA),  etc.,  are  the  probabilities  of  the  joint 
events  in  question,  then  the  expected  value  of  the  decision  will  be 
by  definition, 

EV  =  P(SN.A)  P(N.CA)  P(?N*CA) 
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An  optimum  criterion  would  therefore  be  one  that  would  maximize  this 
expression.  It  can  be  shown  with  suitable  manipulation,  involving 
the  substitution  of  a  priori  and  conditional  probabilities  for  the 
joint  probabilities  in  the  above  expression,  that  maxJ.mizing  of  the 
expected  value  is  equivalent  to  requiring  that 

Povi(^)  ~  is  a  maximum,  where 

dW  N 


0=  1  -  P(SN) 
P(SN) 


(V 


N-CA 


(V  4-  K  X 

SN.A  SN.CA^ 


In  a  set  of  eight  theorems,  Peterson  and  Birdsall  (10)  show  the  re¬ 
lationship  between  an  optimum  criterion  and  the  likelihood  ratio.  Of 
interest  here  are  theorems  1  and  2, 


Theorem  1:  Denote  by  A  the  set  of  points  for  which  the  likelihood 
ratio  -l(x)  ■  Then  A  is  an  optim  m  criterion  (0)  . 

Theorem  2:  If  A  is  an  optimum  criterion  (0) ,  then  the  set  of 

points  in  A  for  which  A(x)<0  has  probability  zero,  and  the 
set  of  points  not  in  for  which  J^(x)>0  has  probability  zero. 

The  proofs  may  be  seen  in  the  reference  article,  Taken  together, 
these  things  show  that  if  the  expected  value  of  the  decision  is  to  be 
maximized  then  those  observations  with  likelihood  ratio  equal  to  or 
greater  than  0  should  be  called  signal  present  and  those  with  likeli¬ 
hood  ratio  less  than  0  should  be  called  signal  absent,  hen*,  e  0  is 
shown  to  be  the  appropriate  minimum  or  cutoff  likelihood  ratio. 

In  summary  that  part  of  the  theory  with  which  we  are  concerned 
deals  with  an  observation,  two  probability  density  functions,  a  set  of 
a  priori  probabilities,  a  set  of  costs  and  values  and  a  rule  for  re- 
Ta'fThg  these  quantities  in  such  a  way  as  to  make  an  optimum  decision 
about  the  observation. 


What  I'anner  hf4S  done  in  his  experiments  is  to  show  that  this 
model  and  ramifications  of  it  can  be  used  to  predict  quantitatively 
the  behavior  of  human  observers  in  situations  admitting  of  a  test  of 
the  model.  He  has  shown  that  observers  can  act  according  to  the  vari¬ 
ous  decl.sion  criteria  described  In  the  theory.  He  has  shown  that  these 
predictions  are  independent  of  particular  experimental,  designs  used. 

The  model  has  been  successfully  used  in  a.'dition  as  well  as  vision  and 
has  been  able  to  achieve  '.uantl ta t i ve  prediction  in  the  case  where  the 
observer  must  recognize  one  of  two  signals  in  noise  instead  of  the 
presence  or  absence  of  a  single  signal . 

It  is  important  to  note  that  Tanner  assumes  that  the  noise  is 
generate^  within  the  observer.  This  opens  the  door,  as  it  were,  to  '■ 
liberal  application  of  the  theory  beyond  simple  situations  v.here  it 
IS  pos'-ible  to  measure  tbe  viic'et  ities  involved.  Vih.o  iukIi  .'•ri’-  Ick-- 

ticn*'  ar..-  U'O'f'u’  rem.sin-  to  I"- 

A  u- ; .L  -■  ro:''!-'’  ‘'-h“  ..'■oi,-.  ol  ■.m  I'r  >ir.  i  s.-l  cb  ,i  1  ( r  i  ;>  1 1 
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While  the  experimental  work  with  the  theory  has  dealt  with  simple 
yes-no  alternatives  or  with  the  recognition  of  one  of  two  possible 
signals,  it  is  possible  to  consider  large  sets  of  alternatives.  The 
theoi’y  requires  that  each  of  these  occur  with  some  a  priori  probabili¬ 
ty  greater  than  zero  and  that  the  sum  of  the  a  prior i  probabilities 
over  all  alternatives  equals  one.  It  can  be  argued  that  by  merely 
considering  the  it"  alternative  as  a  possible  outcome  the  observer 
has  assigned  to  it  some  a  priori  probability  greater  than  zero.  Other¬ 
wise  he  would  not  have  considered  it  as  a  possible  outcome  in  the 
first  place. 

The  a  priori  probabilities  assigned  by  the  man  are  not  neces¬ 
sarily  the'true  a  priori  probabilities.  Instead,  they  represent  the 
observer's  beliefs  concerning  the  true  probabilities.  They  are  based 
upon  his  past  experience,  training  and  understanding.  Obviously  the 
observer  can  only  act  (decide)  on  the  basis  of  subjective  probabilities 
unless  he  is  provided  with  information  about  the  true  probabilities. 

The  theory  does  not  demand  that  he  have  information  about  the  true 
probabilities  but  only  that  a  probability  for  each  alternative  exist. 
When  the  observer  makes  a  decision  he  then  is  exposed,  for  the  first 
time,  to  the  consequences  of  a  discrepancy  between  his  assumed  proba¬ 
bilities  and  the  true  orobabilities ,  since  it  is  then  that  he  becomes 
aware  of  errors.  The  values  and  costs  are  at  this  point  realized, 
and  he  finds  that  he  does  not  realize  his  exvected  values.  /'s  a  re¬ 
sult  of  this  finding  he  may  modify  his  subjective  ensemble  of  proba¬ 
bilities  so  as  to  correspond  more  closely  to  the  true  probabilities. 

The  ability  to  form  likelihood  ratios  is  assured  by  the  noise 
assumption  of  the  theory.  Thus,  for  each  alternative  there  exists  an 
hypothesis,  fj,(x),  the  probability  density  that  if  the  i  th  alternative 
exists  the  observation,  x,  results.  Also,  by  the  noise  assumption  for 
every  i,  f^Cx)  0.  Although,  of  course,  for  many  alternatives  it  may 
be  close  to  zero. 

Finally  Tanner  shows  that  the  theory  may  be  extended  to  con¬ 
sider  complex  alternatives  where  multiple  observations  must  be  made 
to  identify  an  alternative.  Each  complex  alternative,  Aj,  consists 
of  a  sequence  of  simple  alternatives  ...  nj2;  j  ,  ...  aj^. 

Each  simple  alternative  is  associated  with  an  observation,  xi.  The 
set  of  observations,  Xj^,  is  combined  into  a  single  observation  x,  such 
that  for  the  complex  alternative  Aj  there  is  the  probability  density 
function  f^j(x).  This  specifics  the  probability  that  if  the  j^"  com¬ 
plex  alternative  exists  the  particular  sequence  xj,  of  observations  re¬ 
sults.  Hence  for  each  complex  alternative  a  likelihood  ratio  can  bo 
determined  based  on  the  sequence  of  observations  Xj^  and  the  end  result 
is  again  a  single  quantity  as  in  the  case  of  a  simple  alternative. 

Some  interesting  results  are  obtained  by  regarding  ordinary  pilot 
decision  functions  in  the  light  of  this  model.  Consider,  for  example, 
the  target  marker  on  the  search  radar  scope.  The  target  marker  is  a 
small  circle  that  is  positioned  on  the  search  display  by  information 
sent  up  from  the  ground.  It  Is  supposed  to  indicate  where  the  pilot 
should  detect  the  target  on  the  scope.  From  the  point  of  view  of  the 
Tanner  model,  what  the  target  marker  does  is  to  redistribute  the  a  prion 
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probabilities  of  seeing  a  target.  Inside  the  marker  the  a  priori 
probability  is  very  high;  outside  the  marker  the  a  priori  probability 
is  very  low — lower,  in  fact,  than  if  there  were  no  marker.  Inside 
the  marker,  therefore,  the  pilot's  cutoff,  ,  should  be  very  low, 
permitting  him  to  accept  as  target  an  observation  with  a  low  likeli¬ 
hood  ratio.  This,  of  course,  is  what  happens.  An  observation  does 
not  have  to  be  very  greatly  different  from  noise  to  be  accepted  as  a 
target.  If  the  target  marker  were  not  there,  the  same  observation 
would  be  rejected  and  called  "no  target."  Outside  the  target  marker 
the  pilot's  /9  should  be  unusually  high  and  only  observations  with  large 
likelihood  ratio  would  be  accepted  as  target.  This  is  indeed  the  case 
and  it  has  been  shown  that  adding  a  marker  to  the  scope  decreases  the 
probability  of  detecting  a  target  outside  the  marker. 

Consider  a  second  example.  A  study  of  the  detection  and  track¬ 
ing  of  very  weak  targets  was  conducted  with  a  radar  simulator.  Actu¬ 
ally  the  experimenter  presented  no  targets  at  all  most  of  the  time, 
since  he  wished  to  see  if  the  observers  v/ould  accept  noise  or  clutter 
as  targets  under  the  conditions  of  the  experiment.  The  observers  re¬ 
fused  to  do  this  until  the  experimenter  created  what  might  best  be 
called  a  competitive  social  situation  surrounding  the  experiment.  By 
this  device  the  observers  were  induced  to  make  numerous  pseudo  detec¬ 
tions  believing  that  they  were  genuine.  What  the  experimenter  did,  of 
course,  was  greatly  to  inflate  the  value  of  a  detection  while  practical¬ 
ly  eliminating  the  cost  of  a  false  alarm,  thus  permitting  observations 
with  very  small  likelihood  ratios  to  be  accepted  as  a  signal. 

Finally,  consider  a  more  tenuous  example.  A  pilot  is  trying  to 
decide  whether  or  not  to  make  some  particular  flight.  He  is  deterred 
from  doing  so  by  unfavorable  weather.  The  alternatives  between  which 
he  must  decide  are  (1)  the  weather  will  be  good  enough  so  that  a  safe 
flight  can  be  made,  (2)  the  weather  will  be  sufficiently  poor  that  the 
flight  will  be  hazardous.  He  makes  an  observation,  actually  a  complex 
observation,  of  the  weather  from  the  latest  weather  sequence,  direct 
observation,  etc.,  and  (subconsciously)  computes  what  amounts  to  a 
likelihood  ratio  between  the  probability  density  functions  that  (1) 
given  the  first  alternative  this  particular  weather  observation  would 
have  occurred,  and  (2)  given  the  second  alternative  the  same  obser¬ 
vation  would  have  occurred.  His  0  is  set  by  the  a  priori  probabilities, 
which,  in  this  case,  are  given  by  the  weather  forecast,  and  by  costs  and 
values  which  here  are  derived  from  the  importance  of  the  trip  and  the 
cost  of  a  mistake. 
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PROBLEM  AREAS  REQUIRING  EXPERIMENTAL  STUDY  AND 
SOME  SUGGESTED  APPROACHES 


The  purpose  of  this  section  is  to  consider  some  of  the  areas  in 
decision  making  that  require  experimental  and  theoretical  studies,  and 
to  suggest  some  ways  of  approaching  these  problems.  The  theoretical 
development  of  the  area  of  decision  making,  extending  from  the  early 
work  on  the  theory  of  riskless  choices  to  the  recent  work  on  statisti¬ 
cal  decision  theory  and  game  theory  has  a  long  history.  However,  it 
is  only  comparatively  recently  that  any  large  amount  of  significant 
work  has  been  accomplished  in  the  way  of  testing  various  hypotheses 
experimentally  under  well  controlled  conditions. 

At  present  experimental  work  is  being  carried  out  in  the  area  of 
decision  theory  proper,  and  also  in  closely  related  areas  such  as  prob¬ 
lem  solving  and  learning.  It  is  unnecessary  to  review  this  work  here, 
as  reports  of  the  results  are  readily  available  (28,  31,  32,  47,  48,  51, 
79,  82).  It  is  also  beyond  the  scope  of  this  sectTon  of  the  reporT  to 
present  detailed  plans  for  a  comprehensive  program  of  research  on  de¬ 
cision  making.  Several  such  programs  are  in  progress  at  the  present 
time. 


The  need  for-  both  theoretical  and  experimental  studies  in  certain 
specific  areas  became  apparent  during  the  course  of  the  work  on  this 
study  program.  It  is  with  these  areas  that  this  section  is  concerned. 
The  mention  of  particular  areas  in  need  of  research  should  not  be  in¬ 
terpreted  as  implying  that  good  work  is  not  being  done  or  has  not  been 
done  in  these  areas.  In  some  instances  the  areas  are  mentioned  be¬ 
cause  it  is  considered  that  additional  experimental  tests  under  a  wide 
variety  of  conditions  are  needed.  In  other  instances  specific  areas 
are  mentioned  in  order  to  suggest  approaches  that  are  different  from 
ones  heretofore  utilized. 


SUBJECTIVE  PROBABILITIES 

One  of  the  areas  most  in  need  of  additional  experimental  work  is 
that  concerned  with  subjective  probability.  Following  the  distinction 
drawn  by  Edwards  (77)  the  phrase  subjective  probabi I i ty  is  used  here 
as  "a  name  for  a  transformation  on  the  scale  of  mathematical  probabili¬ 
ties  which  is  somehow  related  to  behavior"  rather  than  as  "a  name  for 
a  school  of  thought  about  the  logical  basis  of  mathematical  probability." 
In  the  discussion  of  detection  theory  it  was  pointed  out  that  the  a 
priori  probabilities  utilized  in  decision  making  are  not  necessariTy 
fFe  Frue  probabilities.  The  theory  of  risky  choices  also  required  that 
the  individual  make  estimates  of  the  probability  of  various  outcomes, 
and  the  success  of  utility  maximization  is  largely  dependent  upon  the 
validity  of  these  estimates.  Although  some  research  has  been  done  un 
the  problem  of  the  relationship  of  subjective  probability  to  objective 
probability,  the  area  is  still  largely  unexplored. 

Information  is  needed  about  how  the  individual  develops  the  con¬ 
cept  of  probability,  how  subjective  probability  is  related  to  objective 
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probability  under  a  wide  variety  of  environmental  conditions,  how 
probability  estimates  are  utilized  in  everyday  behavior,  and  the  con¬ 
ditions  under  which  probability  estimates  are  changed  due  to  experi¬ 
ence  with  events.  An  analysis  leading  to  a  theoretical  formulation  of 
the  individual’s  development  and  utilization  of  probability  estimates 
would  be  particularly  valuable.  A  possible  approach  is  suggested  in 
the  following  paragraphs. 

It  is  a  commonplace  observation  that  people  generally  do  not  as¬ 
sign  numerical  probabilities  to  risky  and  uncertain  events  in  everyday 
life.  Except  for  the  phrase  "a  50-50  chance"  most  statements  about 
risky  events  are  non-numer ical .  It  is  probably  safe  to  say  that  most 
people  who  use  the  phrase  "a  50-50  chance"  would  define  this  in  terms 
of  "an  even  chance,"  or  in  some  similar  terminology  rather  than  as  a 
probability  of  0.5  on  the  scale  of  mathematical  probabilities.  It  may 
even  be  the  case  that  persons  who  have  had  formal  training  in  the  math¬ 
ematical  conceptualization  of  probability,  and  who  work  with  the  scale 
of  mathematical  probabilities  formally  do  not  make  use  of  the  fineness 
of  discrimination  possible  with  this  scale  to  any  great  extent  in  their 
everyday  behavior. 

Nevertheless,  it  may  be  observed  that  people  generally  do  recog¬ 
nize  that  events  differ  in  their  probability  of  occurrence.  The  most 
obvious  examples  are  to  be  found  in  activities  that  are  generally  classi¬ 
fied  as  "games  of  chance,"  Almost  anyone  who  has  played  craps  knows 
some  conventional  bets,  for  example,  "even  money,"  "three  to  two,"  etc., 
for  certain  points.  However,  except  for  professional  gamblers,  very  few 
people  know  the  actual  mathematical  probability  of  any  given  point  be¬ 
ing  made.  The  same  thing  is  generally  true  of  the  non-professional 
poker  player.  He  may  know  that  the  rule  says  "you  don't  draw  to  an  in¬ 
side  straight."  He  doesn't  know  what  numerical  probability  should  be 
assigned  to  the  occurrence  of  this  event,  but  if  he  has  played  much 
poker  he  does  have  a  subjective  notion,  based  upon  experience,  that  this 
event  is  not  very  likely  to  occur. 

There  is  also  some  observational  evidence  that  people  use  some 
kind  of  scale  of  subjective  probability  for  referring  to  the  likelihood 
of  occurrence  of  events.  The  English  language  is  replete  with  phrases 
such  as  "almost  a  sure  thing,"  "a  better  than  even  chcnce,"  "highly 
doubtful,"  "practically  impossible,"  etc.  Presumably  these  and  other 
phrases  are  estinjates  of  probability  spanning  the  scale  of  mathematical 
probabilities  from  0  to  1 .  Unfortunately,  because  of  the  plethora  of 
phrases  available,  and  the  wide  variety  of  individual  experiences,  a 
determination  of  the  i*elation  of  specific  phrases  to  points  or  intervals 
on  the  scale  of  mathematical  probabilities  would  have  very  little  gen¬ 
eral  value. 

On  the  other  hand,  it  would  seem  worth  while  to  investigate  the 
number  of  categories  of  subjective  probability  utilized  in  dealing  with 
risky  events.  It  is  interesting  to  specrriate  that  all  individuals  use 
about  the  same  number  of  subjective  categories  even  though  the  names 
used  for  the  categories  differ.  The  results  of  recent  experimental 
determinations  of  the  channel  capacity  for  stimuli  along  simple 
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diraensions  by  the  roethod  of  absolute  judgements  are  suggestive  even 
though  it  is  recognized  that  the  estimation  of  probability  is  not 
equivalent  to  the  absolute  judgement  of  unidimensional  stimuli.  It 
is  tempting  to  expect  that  an  individual  may  use  about  seven  categories, 
corresponding  to  estimates  described  in  phrases  such  as  (1)  event  will 
not  occur,  (2)  almost  impossible,  (3)  less  than  an  even  chanc;,  (4)  an 
even  chance,  (5)  better  than  an  even  chance,  (6)  almost  certain  to  occur, 
and  (7)  absolutely  certain.  Actually,  it  seems  likely  on  the  basis  of 
casual  observation  that  more  than  seven  categories  are  used.  However, 
it  may  be  that  even  if  a  relatively  large  number  of  categories  are  used, 
the  amount  of  information  about  objective  probabilities  transmitted  by 
subjective  estimates  is  no  larger  than  if  a  smaller  number  of  categories 
were  to  be  used.  This  might  result  from  the  fact  that  the  same  subjec¬ 
tive  estimate  is  not  always  made  for  an  objective  probability. 

Whether  seven  or  some  larger  or  smaller  number  of  categories  are 
actually  used,  or  whether  there  is  no  uniformity  among  individuals  or 
within  the  same  individual  under  different  conditions  are,  of  course, 
questions  that  must  be  answered,  not  from  casual  observation  and  arm¬ 
chair  speculation  such  as  led  to  this  discussion,  but  rather  by  the  re¬ 
sults  of  carefully  designed  and  executed  experimentation.  Such  experi- 
mencs  should  be  designed  to  determine  not  only  whether  the  individual 
categorizes  probability  estimates  at  the  level  of  verbal  behavior,  but 
also  whether  his  non-verbal  behavior  in  situations  involving  probabilis¬ 
tic  events  shows  evidence  of  categorization  of  probability  estimates. 


COSTS  AND  VALUES 

Another  area  in  which  little  is  known  about  how  the  human  behaves 
is  that  one  concerned  with  the  assignment  of  utilities  or  costs  and 
values  to  outcomes.  The  experimental  work  in  this  area  has  been  con¬ 
cerned  for  the  most  part  with  the  measurement  of  utility  for  money.  A 
few  studies  that  have  not  utilized  money  payoffs  have  used  commodities 
for  which  a  monetary  equivalent  is  generally  known.  In  the  experimental 
work  on  the  decision  making  theory  of  detection  Tanner  and  his  associates 
have  used  money  costs  and  values  for  false  alarms,  misses,  hits,  and  re¬ 
jections.  Although  this  was  expedient  for  the  preliminary  experimental 
work,  more  needs  to  be  known  about  the  scales  of  costs  and  values  in  a 
practical  detection  situation.  It  seems  highly  doubtful  that  a  radar 
observer,  for  example,  conceptualizes  the  value  of  a  detection  and  the 
cost  of  a  false  alarm  or  a  miss  in  terms  of  dollars.  Nevertheless, 
there  presumably  is  some  ordering  of  the  utility  of  outcomes.  As  was 
the  case  with  subjective  probability  estimates,  it  is  tempting  to  specu¬ 
late  that  an  individual  works  with  a  relatively  small  number  of  cate¬ 
gories  of  costs  and  values  rather  than  with  a  finely  divided  scale  in¬ 
volving  many  discriminable  intervals. 


EFFICIENCY  OF  DECISION  MAKING 

A  number  of  studies  within  the  framework  of  detection  theory 
have  been  concerned  with  estimating  the  efficiency  of  the  human  observer 
as  a  decision  maker.  This  is  done  by  comparing  the  performance  of  the 
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human  observer  with  that  of  an  "ideal"  observer  as  a  standard.  The 
perfoi'mance  of  the  ideal  observer  is  expressed  as  a  ratio  of  proba- 

of  detection  to  probability  of  false  alarm  for  different  values 
of  0  ,  the  cutoff  criterion,  and  for  a  given  signal-to-noise  ratio. 

This  is  a  mathematical  computation  that  has  been  worked  out  in  a  number 
of  special  cases,  for  example  in  the  cases  of  a  signal  known  exactly 
or  a  signal  known  except  for  phase,  in  the  presence  of  Fourier  series 
band— limited  white  Gaussian  noise.  The  performance  of  such  an  observer 
is  optimum  in  the  sense  that  all  of  the  information  in  the  input  is 
used  in  making  the  decision.  The  huBian  observer  is  presented  with  the 
same  signal  and  noise  and  his  detection  and  false  alarm  rates  are 
measured  in  an  experiment.  Since  the  human's  performance  must  be  less 
than  or  equal  to  that  of  the  ideal  observer  the  decrement,  if  any,  is 
measured  by  computing  the  amount  that  the  signal  energy  would  have  to 
be  attenuated  in  order  for  the  ideal  observer  to  yield  the  same  per¬ 
formance  as  the  man.  The  ratio  of  this  energy  to  the  actual  energy  in 
the  signal  is  a  measure  of  the  human  observer’s  efficiency.  The  noise 
power,  of  course,  is  kept  the  same  throughout. 

This  is  one  example  of  the  kind  of  research  that  is  being  done 
in  decision  theory.  Along  with  others  of  its  kind  it  depends  upon 
measurement  of  and  control  over  signal  energy  and  noise  power.  Some 
of  the  signals  with  which  pilots  deal  are  of  this  kind.  It  is  possible 
to  measure  signal  energy  and  noise  power  and  to  conduct  experiments  in 
reasonaoly  strict  interpretation  of  the  model.  With  most  signals  of 
interest,  however,  this  is  not  possible.  It  is  meaningless,  for  example, 
to  talk  of  the  signal  energy  in  an  aircraft  instrument  reading.  The 
model  is  able  to  handle  such  variables  but  the  design  of  experiments 
under  these  circvimstances  becomes  difficult. 


APPLICATION  TO  PILOT  DECISION  MAKING 

According  to  Marill  (7)  a  detection  situation  exists  whenever 
the  following  five  conditions  are  true: 

1.  On  each  "trial"  one  or  the  other  of  two  mutually  exclusive 
states  S  or  ^  obtains.  The  a  priori  pi’obability  P(S)  of  the  occur¬ 
rence  of  S  is  assumed  to  be  FTixed” given  number, 

2.  On  each  trial  the  subject  (the  "detector")  brings  about  the 
occurrence  of  one  or  the  other  of  two  mutually  e:vclusive  events  ("re¬ 
sponses")  R  or  It, 

3.  Event  ,  O2 ,  Q3 ,  or  occurs  on  a  given  trial  according 

as  the  joint  event  S*R,  S*It,  or  also  occui's  on  that  trial. 

4.  The  occurrence  of  the  event  (i  =  1,  ...,  4)  has  a  speci¬ 
fiable  value,  Vj^  (the  "payoff  value"  or  "payoff")  for  the  subject. 

5.  The  following  inequalities  hold:  V,  >  V,-,;  V,  >  V9. 

i  O  ^4  A-» 

In  the  case  of  pilot  decision  making,  introspection  and  analysis 
suggest  that  a  process  somewhat  like  the  following  may  take  place.  The 
pilot  is  attempting  to  diagnose  the  state  of  his  aircraft  with  respect 
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to  some  variable,  say  geographic  position.  The  pilot  comes  to  the 
decision  situation,  and  this  is  important,  with  an  hypothesis  concern¬ 
ing  what  the  state  is.  He  makes  an  observation  such  as  the  reading  of 
a  set  of  instruments.  He  then  concludes,  or  decides,  that  the  hypo¬ 
thesis  is  correct  or  incorrect. 

The  state  S  is  the  state  that  the  instrument  readings  do  in  fact 
confir**;  the  pilot’s  hypothesis  and  the  state  ¥  is  that  they  do  not. 
P(S/  is  therefore  the  a  priori  probability  that  the  pilot's  hypothesis 
is  correct. 


The  response  R  is  the  conclusion  by  the  pilot  that  his  hypothesis 
has  been  confirmed;  IT  that  it  has  not  been  confirir'-d. 


The  events  are  therefore; 


Ql 


The  hypothesis  is  confirmed  and  the  pilot  concludes  it  has 
been  confirmed. 


Q2  =  The  hypothesis  is  not  confirmed  and  the  pilot  concludes  it 
is  confiimed, 

Qg  =  The  hypothesis  is  confirmed  and  the  pilot  concludes  it  is 
not  confirmed. 


^4 


The  hypothesis  is  not  confirmed  and  the  pilot  concludes  it 
is  not  confirmed. 


It  is  reasonable  to  suppose  that  each  has  some  value  to  the 
pilot,  although  it  is  obviously  difficult  to  assign  numbers  to  these. 
Nevertheless  clause  No.  5  appears  to  hold;  that  is,  it  is  reasonable 
that 


V,  >  and  V  >  V  . 

13  4  2 


Ignoring  the  for  the 
form  for  deciding  iii  favor  of 
is  when  ?(?)Pg  (x)  >  PCF) 


moment  it  will  be  roDalled  that  the  general 
alternative  S,  i.e.,  making  the  response  R, 
<x)  ,  or  whenever  the  likelihood  ratio 


l(x) 


>  P(^) 

P^(x)  ^  P(?) 


where  x  stands  for  the  observation,  in  this  case  the  instrument  readings. 
The  term  Pg(x)  may  be  interpreted  as  follows:  Pc(x)  is  the  likelihood 

in  the  pilot's  own  mind  that  the  observation  he  has  made  could  belong  to 
the  set  of  those  observatioi.s  that  confirm  his  hypothesis.  Obviously, 
in  order  to  give  substance  to  this  interpre  t.ation  it  is  necessary  to  as- 
stime  that  the  pilot  introduces  uncertainty  somewhere  in  the  obsorvatiini 
process.  Those  who  are  pilots  or  have  studied  pilot  behavior  have  no 
difficulty  in  making  this  assumption.  It  would  appear,  therefore,  that 
ordinary  pilot  decision  making  can  be  reasonably  interpreted  in  terms  of 
de tec tion , theory .  It  remains  to  be  determined  how  useful  the  interpre¬ 
tation  can  be. 
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The  reader  can  try  the  exercise  of  formulating  various  common 
pilot  decision  problems  in  terms  of  the  theory  to  see  if  any  insight  is 
gained.  For  example  there  is  the  problem  of  the  false  alarm,  i,e,,  the 
"fat,  dumb  and  happy"  pilot  who  unaccountably  acts  in  a  way  that  is  com¬ 
pletely  incompatible  with  the  inputs  he  is  getting.  The  event  of  a  miss 
is  equally  serious.  Here  the  pilot’s  hypothesis  is  correct  but  he  in¬ 
terprets  the  observation  as  indicating  a  rejection  of  it,  necessitating 
the  formulation  of  a  new  hypothesis  which  must  necessarily  be  wrong. 

This  in  turn  may  lower  the  threshold  for  a  subsequent  false  alarm.  Such 
behavior  is  the  cause  of  many  accidents.  There  is  the  difference  in  de¬ 
cision  behavior  that  is  observed  when  the  pilot’s  inputs  are  coded  in 
different  ways,  for  example,  contact  versus  instrument  flight  or  pic¬ 
torial  versus  symbolic  instrument  displays.  This  leads  to  interesting 
speculation  concerning  how  the  pilot  generates  uncertainty  in  the  ob¬ 
servation  process  and  whether  this  may  depend  upon  the  relationship  be¬ 
tween  the  form  of  the  input  and  the  form  in  which  the  pilot’s  hypothesis 
is  stored  in  his  memory.  The  assumption  that  the  pilot  comes  to  the 
decision  process  with  an  hypothesis  to  be  tested  is  basic  to  the  present 
interpretation  of  the  model  and  should  itself  be  subjected  to  experi¬ 
mental  verification.  Many  other  implications  of  the  model  can  be 
imagined  and  the  difficulties  of  subjecting  them  to  experimental  test 
must  be  overcome  in  subsequent  work. 
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FINAL  REMARKS 


The  bulk  of  this  s'-dy  has  been  concerned  with  a  detailed  analy¬ 
sis  of  the  pilot task  in  a  modern  and  quite  sophisticated  weapon  sys¬ 
tem.  The  details  of  this  work  have  not  been  reported  here,  partly  be¬ 
cause  of  security  considerations  but  also  because  no  useful  purpose 
would  be  served  by  doing  so.  This  analysis  shewed  clearly,  in  the 
opinion  of  the  investigators,  that  the  pilot’s  decision  functions. in 
this  system  at  least,  are  concerned  with  the  diagnosis  of  the  state  of 
the  system  and  only  rarely  with  the  choice  of  a  course  of  action  to 
pursue.  In  no  sense  does  the  pilot  simply  jump  in  his  airplane,  take 
off,  and  then  decide  what  to  do  as  he  goes  along.  The  mission,  of 
course,  is  carefully  planned  in  advance  but  the  courses  of  ac^-^on  open 
to  the  pilot  are  largely  built  into  the  system.  These  are  the  modes  of 
operation  of  which  the  system  is  capable.  A  mode  is  provided  for  each 
major  state  in  which  it  is  expected  the  system  will  find  itself.  The 
pilot  is  then  to  determine  what  the  state  is.  Having  done  this  he  then 
adjusts  the  equipment  to  operate  in  the  mode  specified  in  advance  for 
that  state.  His  decision,  then,  is  a  diagnosis  —  a  detection  and  rec¬ 
ognition  of  the  state  of  the  system. 

These  conclusions  led  to  some  speculation  concerning  the  nature 
of  weapon  systems  in  general.  It  was  postulated  that  all  weapon  sys¬ 
tems,  both  manned  and  unmanned,  have  what  might  be  termed  a  program. 

The  program  is  essentially  a  list  of  events  that  may  happen  to  a  system 
during  a  mission,  together  with  a  set  of  adaptive  responses,  each  de¬ 
signed  to  compensate  for  the  effects  of  an  event  so  that  the  mission 
will  be  successful  despite  the  event.  Some  comparisons  between  manned 
and  unmanned  systems  were  made  in  terms  of  their  programs.  It  was 
shown  that  the  man  is  in  the  system  primarily  to  detect  and  recognize 
the  occurrence  of  events  that  it  is  difficult  to  detect  automatically. 
But  once  in  the  system  he  is  given  many  other  duties  as  a  matter  of 
economy. 

Next,  a  brief  review  of  the  more  prominent  decision  theories  was 
made.  Most  time  was  spent  on  the  model  of  Tanner’s,  because  this  seemed 
to  coirespond  most  closely  with  what  the  pilot  does  in  operating  a 
weapon  system.  The  difficulties  involved  in  obtaining  the  required 
quantitative  information  for  use  of  the  model  were  indicated.  Some 
suggested  approaches  towards  solving  this  problem  were  made.  Finally 
a  general  form  for  application  of  the  model  to  pilot  decision  problems 
was  given. 

What  was  not  done,  although  it  would  have  been  desirable,  was  to 
provide  a  careful  look  at  the  impl icat j.ons  of  using  the  model  i  i  future 
studies  of  pilot  decision  functions.  In  those  cases  of  interes'  it 
would  appear  that  all  important  variables  are  subject  i.'o  and  use  of  the 
model  may  simply  end  in  tautology.  On  the  other  hand  it  is  not  impos¬ 
sible  that  a  way  around  this  may  be  found.  It  would  appf  ■  that  this 
should  be  a  subject  for  a  separate  study. 

Pra.ctically  nothing  has  been  said  about  displays  and  controls  for 
future  systems,  although  it  would  be  possible  to  speculate  at  some 
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length  using  concepts  stemming  from  decision  theory.  For  example, 
there  may  be  considerable  merit  to  the  idea  of  displaying  probability 
estimates  to  the  operator  where  such  information  is  available.  In¬ 
deed,  such  a  display  was  proposed  at  one  time  by  Hughes  Aircraft  Company 
but  was  ultimately  abandoned  because  of  construction  difficulties.  On 
the  other  hand  it  is  possible  to  err  in  the  opposite  direction  and  pre¬ 
sent  probabilistic  information  as  if  it  were  certain.  This  can  lead  to 
unfortunate  decisions  by  the  pilot. 
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